-- Logs begin at Tue 2022-10-04 04:54:18 UTC, end at Tue 2022-10-04 06:02:18 UTC. -- Oct 04 05:23:01.339880 dv-cinder-849143-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 04 05:23:01.347096 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 04 05:23:01.349573 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 4 05:23:01 2022] *** Oct 04 05:23:01.349573 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 04 05:23:01.349573 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 04 05:23:01.349573 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: nodename: dv-cinder-849143-10-iscsi Oct 04 05:23:01.349573 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: machine: x86_64 Oct 04 05:23:01.349573 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: clock source: unix Oct 04 05:23:01.349573 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: pcre jit disabled Oct 04 05:23:01.349573 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: detected number of CPU cores: 4 Oct 04 05:23:01.350179 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: current working directory: / Oct 04 05:23:01.350179 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: detected binary path: /usr/bin/uwsgi-core Oct 04 05:23:01.350179 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: your processes number limit is 47680 Oct 04 05:23:01.350179 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: your memory page size is 4096 bytes Oct 04 05:23:01.350179 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: detected max file descriptor number: 2048 Oct 04 05:23:01.350179 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: lock engine: pthread robust mutexes Oct 04 05:23:01.350179 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: thunder lock: enabled Oct 04 05:23:01.350179 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 04 05:23:01.350179 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 04 05:23:01.389765 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: Python main interpreter initialized at 0x5639e9a5b9c0 Oct 04 05:23:01.389765 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: python threads support enabled Oct 04 05:23:01.389765 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: your server socket listen backlog is limited to 100 connections Oct 04 05:23:01.389765 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: your mercy for graceful operations on workers is 90 seconds Oct 04 05:23:01.390100 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: mapped 403077 bytes (393 KB) for 2 cores Oct 04 05:23:01.390245 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: *** Operational MODE: preforking *** Oct 04 05:23:01.390514 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: *** uWSGI is running in multiple interpreter mode *** Oct 04 05:23:01.390514 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: spawned uWSGI master process (pid: 81472) Oct 04 05:23:01.390727 dv-cinder-849143-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 04 05:23:01.394192 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: spawned uWSGI worker 1 (pid: 81480, cores: 1) Oct 04 05:23:01.394192 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: spawned uWSGI worker 2 (pid: 81481, cores: 1) Oct 04 05:23:01.394192 dv-cinder-849143-10-iscsi devstack@keystone.service[81472]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 04 05:23:05.727583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 05:23:05.727583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: from cryptography.utils import int_from_bytes Oct 04 05:23:05.728426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 05:23:05.728426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: from cryptography.utils import int_from_bytes Oct 04 05:23:05.728559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 05:23:05.728559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: from cryptography.utils import int_from_bytes Oct 04 05:23:05.728559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 05:23:05.728559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: from cryptography.utils import int_from_bytes Oct 04 05:23:05.781659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 05:23:05.781859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 05:23:05.781997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 05:23:05.782101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] command line args: [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 05:23:05.782221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 05:23:05.782332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] command line args: [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 05:23:05.782411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 05:23:05.782515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ================================================================================ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 05:23:05.782603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 05:23:05.782774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ================================================================================ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 05:23:05.782856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] admin_token = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.783022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config_dir = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.783104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] admin_token = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.783214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.783303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_dir = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.783400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] config_source = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.783481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.783582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] debug = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.783658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] config_source = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.783782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.784030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] debug = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.784030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.784162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.784228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.784312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.784379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.784585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.784675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.784838 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.784932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.785072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.785151 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.785253 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_config_append = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.785330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_config_append = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.785430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.785499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.785615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_dir = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.785679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_dir = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.785794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.785873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.786024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.786215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.786318 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.786383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.786538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.786656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.786788 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.786917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.787022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.787139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.787287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.787417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.787518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.787620 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.787722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.787817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.787911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.788009 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.788114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.788209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.788302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.788400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.788501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.788600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.788724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.788839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.788971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.789073 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.789246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.789360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.789734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.789851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.789948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] publish_errors = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.790050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] publish_errors = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.790185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.790285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.794230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.794347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.794490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.794569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.794641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.794734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.794806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.794886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.794963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] standard_threads = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] standard_threads = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_journal = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_journal = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_json = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_json = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_stderr = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.795959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_stderr = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.796027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] use_syslog = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.796102 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] use_syslog = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.796172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.796248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 05:23:05.796319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.796388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.796457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.796526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.796670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.796754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.796825 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.796894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.796962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797177 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797747 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797823 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.797969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.798997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799135 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.799974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.800909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.801964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802349 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.802980 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.803066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.803217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.803290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.803467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.803533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.803732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.803819 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.803932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.804053 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.804166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.804277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.804390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.804511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.804634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.804739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.804858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.804972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.805074 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.805185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.805311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.805416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.805527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.805639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.805778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.805882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.806002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.806116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.806244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.806366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.806486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.806599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.806724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.806829 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.806940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.807053 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.807170 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.807290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.807411 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.807527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.807647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.807774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.807894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.808007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.808119 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.808244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.808358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.808470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.808582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.808694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.808797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.808916 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.809029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.809141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.809252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.809370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.809483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.809602 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.809721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.809836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.809947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.810058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.810231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.810421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.810537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.810649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.810768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.810883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.811003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.811117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.811230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.811387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.811623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.811743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.811743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.811979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.812106 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.812220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.812340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.812453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.812566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.812678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.812799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.812913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.813027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.813139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.813257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.813370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.813490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.813604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.813723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.813836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.813948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.814062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.814201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.814318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.814431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.814543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.814655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.814768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.814888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.815001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.815115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.815227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.815339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.815451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.815572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.815685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.815799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.815911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.816023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.816143 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.816265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.816378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.816491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.816603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.816724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.817584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.817713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.817828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.817941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.818053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.818178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.818295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.818419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.818542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.818656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.818797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.818920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.external = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.819076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.external = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.819192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.819296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.819402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.819501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.819603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.819700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.819795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.password = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.819898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.password = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.819996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] auth.token = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.820091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] auth.token = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.820186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.820280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.820375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.820478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.820574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.820675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.820770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.820865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.820959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.821062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.821165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.821271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.821369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.821472 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.821567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.821670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.821800 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.821911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.822048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.822235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.822339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.822895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.823009 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.823126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.823240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.823353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.823468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.823594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.823709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.823829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.823944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.824058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.824058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.824277 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.824398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.824513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.824626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.824739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.824852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.825039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.825156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.825277 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.825391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.825504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.825618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.825739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.825854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.825967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.826080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.826210 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.826325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.826455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.826571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.826686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.826805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.826919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.827032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.827154 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.827269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.827382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.827495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.827609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.827724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.827846 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.827960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.828073 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.828213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.828337 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.828451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.828573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.828573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.828787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.828787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.828997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.829110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.829231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.829336 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.829447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.829560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.829673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.829795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.829910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.830023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.830171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.830288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.830412 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.830534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.830649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.830768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.830883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.830996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.831109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.831230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.831344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.831457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.831570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.831691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.831807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.831929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.832043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.832156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.832269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.832381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.832525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.833165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.833284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.833396 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.833512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.833625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.833736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.833856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.833971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.834083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.834212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.834395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.834515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.834642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.834758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.834871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.834985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.835099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.835218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.835341 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.835457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.835569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.835683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.835796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.835915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.836038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.836153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.836266 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.836379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.836492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.836604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.836725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.836840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.836960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.837075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.837188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.837300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.837421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.837535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.837648 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.837761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.837873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.837986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.838108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.838266 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.838390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.838504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.838617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.838729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.838895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.839031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.839146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.839262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.839375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.839488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.839610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.839725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.839839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.839976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.840128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.840236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.840359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.840460 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.840573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.840685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.840798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.840911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.841032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.841146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.841259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.841372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.841485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.841598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.841719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.841833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.841946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.842059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.842190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.842318 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.842441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.842555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.842668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.842781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.842893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.843015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.843130 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.843243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.843356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.843507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.843624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.843746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.843860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.843973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.844086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.844198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.844311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.844433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.844602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.844719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.844831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.844944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.845086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.845209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.845324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.845437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.845558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.845672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.845785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.845906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.846048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.846180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.846282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.846377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.846473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.851580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.851714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.851792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.851909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.851986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852361 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.852937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.853952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854788 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.854946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.855959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.856861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.857153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.857339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.857598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.857839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.858074 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.858338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.858582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.858699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.858821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.859079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.859353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.859568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.859808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.860045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.860281 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.860522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.860769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.860989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.861122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.861255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.861387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.861502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.861582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.861740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.861818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.861984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.862071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.862254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.862368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.862505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.862704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.862839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.862966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.863080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.863197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.863360 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.863523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.864200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.864334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.864445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.864571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.864702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.864928 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.865046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.865191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.865320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.865435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.865564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.865692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.865814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.865939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.866070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.866237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.866358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.866479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.866603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.866726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.866892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.867041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.867193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.867314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.867314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.867542 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.867670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.867790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.867904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.868024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.868148 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.868283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.868421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.868587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.868711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.868836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.868990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.869109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.869235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.869352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.869465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.869580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.869693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.869806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.869928 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81480) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 05:23:05.870044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.870044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.870270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.870382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.870633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.870884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.871123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.871368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.871613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.871863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.872102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.872342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.872582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.872829 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.873102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.873332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.873576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.873843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.874103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.874371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.874634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.874855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.875097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.875332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.875566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.875800 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.876035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.876278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.876513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.876756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.876991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.877239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.877472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.877717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.877948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.878208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.880312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 05:23:05.880312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 05:23:05.899306 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.899544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.899772 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.903589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.903823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.904048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.908447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.912281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.939665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.939895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.940159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:05.940389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.008049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.008207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.059927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.060134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.060223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.060373 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.226760 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.226987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.227127 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.227220 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.227364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.227476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.227589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.227683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.227803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.227886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.228018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.228235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.254131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.254301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.255027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.255125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.255258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.255342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.255480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.255579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.255694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.255784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.255904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.255983 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.256112 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.256191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.267212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.267468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.279656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.279886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.280038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.280121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.280273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.280354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.280496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.280578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.280705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.280794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.280916 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.280996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.281126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.281341 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81480) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 05:23:06.310031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.310257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 05:23:06.310480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.310596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.310678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 05:23:06.310793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 05:23:06.311021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.311105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.311223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 05:23:06.311304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.311498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.311577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.311763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.311844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.312026 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.312285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.321628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.321903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.322156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.322240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.322450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.322532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.322723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.323006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.329686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.329961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.332598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.332879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.332975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.333160 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.333239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.333439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.333519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.333740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.333874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.333994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.334072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.334257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.334354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.334533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.334614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.334803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.334891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.335077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.335165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.335343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.335424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.335633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.358101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.358632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.359572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.359850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.360134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.360421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.365117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.365394 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.365613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.365696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.365898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.366202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.371923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.372207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.372480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.372769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.377748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.378045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.378359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.378444 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.378648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.378731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.378927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.379010 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.379206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.379327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.379507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.379588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.379777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.379860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.380102 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.380376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.394883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.395184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.395708 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.395991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.398946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.399765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.402079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.402392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.402991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.403293 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.406468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.407409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.409759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.410456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.413130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.413428 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.413713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.413999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.414203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.414373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.414511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.414655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.414801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.415092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.415376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.415657 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.430601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.430886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.431150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.431410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.431705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.431996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.432267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.432536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.438115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.439357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.439652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.440871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.441161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.441435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.442765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.443043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.444394 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.444678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.444958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.445237 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.445637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.445740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.445953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.446264 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.446551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.446958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.456640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.456947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.457246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.457553 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.457939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.458022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.458256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.458345 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.458541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.458623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.458877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.459273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.459569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.459861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.474101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.475573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.476985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.478537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.479853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.481336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.481417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.482864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.486519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.486697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.486697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.486839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.486839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.486839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.490426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.491866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.491958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.493304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.494742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.495138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.495425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.495744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.496193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.496597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.496888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.497171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.503374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.503681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.503961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.505059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.505364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.505649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.510805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.511087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.511369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.511647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.511921 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.512193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.512466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.512557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81480) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 05:23:06.512747 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.512831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.513017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.513101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.513295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.513429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.513586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.513678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.513843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.513939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.514105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.514208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.514398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.514481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.514667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.518758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.518758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.518758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.518758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.518758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81480) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 05:23:06.548223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639e9a5b9c0 pid: 81481 (default app) Oct 04 05:23:06.550337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5639e9a5b9c0 pid: 81480 (default app) Oct 04 05:23:06.552096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec7a7d-d2dd-4ff5-9a93-0d1df43ecd39 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:06.552383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec7a7d-d2dd-4ff5-9a93-0d1df43ecd39 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:06.552654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec7a7d-d2dd-4ff5-9a93-0d1df43ecd39 None None] PATH_INFO: `/v3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:06.556762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Tue Oct 4 05:23:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:14.172066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a12bd-346c-4227-b4da-0de909493021 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:14.172647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a12bd-346c-4227-b4da-0de909493021 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:14.172647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a12bd-346c-4227-b4da-0de909493021 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:14.176789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:14.181930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:14.182378 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:14.182518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:14.254548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: INFO dbcounter [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] Registered counter for database keystone Oct 04 05:23:14.274068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG oslo_db.sqlalchemy.engines [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81481) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 05:23:14.280357 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writer thread running {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 05:23:14.460951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG passlib.handlers.bcrypt [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81481) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 05:23:14.461254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG passlib.handlers.bcrypt [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81481) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 05:23:14.518656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:14.520818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:14.645562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 05:23:15.100051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a12bd-346c-4227-b4da-0de909493021 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:15.169272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: INFO dbcounter [None req-4a1a12bd-346c-4227-b4da-0de909493021 None None] Registered counter for database keystone Oct 04 05:23:15.190568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG oslo_db.sqlalchemy.engines [None req-4a1a12bd-346c-4227-b4da-0de909493021 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81480) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 05:23:15.197651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writer thread running {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 05:23:15.322278 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a12bd-346c-4227-b4da-0de909493021 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:15.326542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b66d52-e9a7-4949-bdf2-6aeec9c7318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:15.327187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b66d52-e9a7-4949-bdf2-6aeec9c7318d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:15.327464 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b66d52-e9a7-4949-bdf2-6aeec9c7318d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:15.327736 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b66d52-e9a7-4949-bdf2-6aeec9c7318d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:15.335858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b66d52-e9a7-4949-bdf2-6aeec9c7318d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:15.386441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e1b66d52-e9a7-4949-bdf2-6aeec9c7318d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 05:23:15.386787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b66d52-e9a7-4949-bdf2-6aeec9c7318d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:15.395975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Tue Oct 4 05:23:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:23:15.401999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:15.418827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894d20-dbc7-40e9-aa7d-c1c8f7ed889a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:15.423191 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d736047-2c9c-4eb7-80c1-7cc4bd434ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:15.424236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d736047-2c9c-4eb7-80c1-7cc4bd434ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:15.424516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d736047-2c9c-4eb7-80c1-7cc4bd434ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:15.424782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d736047-2c9c-4eb7-80c1-7cc4bd434ecd admin admin] PATH_INFO: `/v3/projects/4c2b6383659b48fcabc56888b6bfb190` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:15.426996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d736047-2c9c-4eb7-80c1-7cc4bd434ecd admin admin] RBAC: Authorizing `identity:get_project(project_id=4c2b6383659b48fcabc56888b6bfb190)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:15.478573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1d736047-2c9c-4eb7-80c1-7cc4bd434ecd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 05:23:15.478927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d736047-2c9c-4eb7-80c1-7cc4bd434ecd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:15.483528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Tue Oct 4 05:23:15 2022] GET /identity/v3/projects/4c2b6383659b48fcabc56888b6bfb190 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:17.569201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b756fbb-3b41-48c5-a754-7561600ef787 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:17.569713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b756fbb-3b41-48c5-a754-7561600ef787 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:17.569853 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b756fbb-3b41-48c5-a754-7561600ef787 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:17.573830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:17.579105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917a2a4-6298-406f-8546-687e45cf6a00 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:17.579529 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917a2a4-6298-406f-8546-687e45cf6a00 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:17.579647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917a2a4-6298-406f-8546-687e45cf6a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:17.599645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5917a2a4-6298-406f-8546-687e45cf6a00 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:17.601148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5917a2a4-6298-406f-8546-687e45cf6a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:17.608811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 05:23:17.870404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b756fbb-3b41-48c5-a754-7561600ef787 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:17.883156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b756fbb-3b41-48c5-a754-7561600ef787 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:17.888584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5ce6b-d827-431b-b680-e873f4d36478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:17.889200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5ce6b-d827-431b-b680-e873f4d36478 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:17.889938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5ce6b-d827-431b-b680-e873f4d36478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:17.889938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5ce6b-d827-431b-b680-e873f4d36478 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:17.898195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5ce6b-d827-431b-b680-e873f4d36478 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:17.899972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5ce6b-d827-431b-b680-e873f4d36478 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:17.908634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Tue Oct 4 05:23:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:23:17.914866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917a2a4-6298-406f-8546-687e45cf6a00 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:17.923941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917a2a4-6298-406f-8546-687e45cf6a00 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:17.928307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce740606-64d0-4ab4-96e6-d4e1dd17c091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:17.928923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce740606-64d0-4ab4-96e6-d4e1dd17c091 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:17.929199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce740606-64d0-4ab4-96e6-d4e1dd17c091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:17.929470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce740606-64d0-4ab4-96e6-d4e1dd17c091 admin admin] PATH_INFO: `/v3/users/4444aa1a4685411fad68fe8a8f2d8f11` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:17.930861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce740606-64d0-4ab4-96e6-d4e1dd17c091 admin admin] RBAC: Authorizing `identity:get_user(user_id=4444aa1a4685411fad68fe8a8f2d8f11)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:17.932665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce740606-64d0-4ab4-96e6-d4e1dd17c091 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:17.934178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Tue Oct 4 05:23:17 2022] GET /identity/v3/users/4444aa1a4685411fad68fe8a8f2d8f11 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:20.140718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6721c8-c67b-4ea9-a559-d83d8552daa2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:20.141241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6721c8-c67b-4ea9-a559-d83d8552daa2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:20.141241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6721c8-c67b-4ea9-a559-d83d8552daa2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:20.142593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:20.148376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a6ca-c791-41a4-9306-8b514f547d26 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:20.148765 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a6ca-c791-41a4-9306-8b514f547d26 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:20.148930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a6ca-c791-41a4-9306-8b514f547d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:20.158747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ad576-05c0-427b-bdf7-6ad539ae56ef None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:20.159035 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ad576-05c0-427b-bdf7-6ad539ae56ef None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:20.159314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ad576-05c0-427b-bdf7-6ad539ae56ef None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:20.161215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:23:20.166111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78766f8d-3156-4192-bf38-75eb1be2cf02 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:20.166426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78766f8d-3156-4192-bf38-75eb1be2cf02 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:20.166704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78766f8d-3156-4192-bf38-75eb1be2cf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:20.167569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3981a6ca-c791-41a4-9306-8b514f547d26 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:20.168770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3981a6ca-c791-41a4-9306-8b514f547d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:20.208481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG passlib.handlers.bcrypt [None req-78766f8d-3156-4192-bf38-75eb1be2cf02 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81480) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 05:23:20.208864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG passlib.handlers.bcrypt [None req-78766f8d-3156-4192-bf38-75eb1be2cf02 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81480) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 05:23:20.248906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:20.257882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-78766f8d-3156-4192-bf38-75eb1be2cf02 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:20.259548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-78766f8d-3156-4192-bf38-75eb1be2cf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:20.359238 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:23:20.517240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a6ca-c791-41a4-9306-8b514f547d26 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:20.542156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:20.617252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a6ca-c791-41a4-9306-8b514f547d26 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:20.621647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78766f8d-3156-4192-bf38-75eb1be2cf02 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:20.651057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c58d51-a1d8-459f-8f38-680449a61c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:20.651702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c58d51-a1d8-459f-8f38-680449a61c01 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:20.651982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c58d51-a1d8-459f-8f38-680449a61c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:20.652253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c58d51-a1d8-459f-8f38-680449a61c01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:20.663033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c58d51-a1d8-459f-8f38-680449a61c01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d4c58d51-a1d8-459f-8f38-680449a61c01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:20.671754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:20.672646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:20.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:23:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:23:20.678597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c58d51-a1d8-459f-8f38-680449a61c01 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:20.721906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78766f8d-3156-4192-bf38-75eb1be2cf02 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:20.724324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:20.757710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:20.759066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baf362-8ae5-45f8-9ba4-1d0416be3105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:20.759924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baf362-8ae5-45f8-9ba4-1d0416be3105 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:20.760323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baf362-8ae5-45f8-9ba4-1d0416be3105 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:20.760600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baf362-8ae5-45f8-9ba4-1d0416be3105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:20.778205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c58d51-a1d8-459f-8f38-680449a61c01 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:20.808034 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b3948-6851-4485-9912-becda52942c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:20.808643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b3948-6851-4485-9912-becda52942c8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:20.808924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b3948-6851-4485-9912-becda52942c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:20.809193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b3948-6851-4485-9912-becda52942c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:20.809854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b3948-6851-4485-9912-becda52942c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:20.810179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b3948-6851-4485-9912-becda52942c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:20.811563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b3948-6851-4485-9912-becda52942c8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:20.823647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:20.826725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09baf362-8ae5-45f8-9ba4-1d0416be3105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:20.828310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09baf362-8ae5-45f8-9ba4-1d0416be3105 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:20.829194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b3948-6851-4485-9912-becda52942c8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:20.928749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b3948-6851-4485-9912-becda52942c8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:20.929358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:23:20.934968 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baf362-8ae5-45f8-9ba4-1d0416be3105 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:20.937588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:20.961818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316856c0-7b36-49f3-84c2-a5808dc974fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:20.962475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316856c0-7b36-49f3-84c2-a5808dc974fb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:20.962752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316856c0-7b36-49f3-84c2-a5808dc974fb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:20.963022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316856c0-7b36-49f3-84c2-a5808dc974fb None admin] PATH_INFO: `/v3/users/4444aa1a4685411fad68fe8a8f2d8f11` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:20.964142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316856c0-7b36-49f3-84c2-a5808dc974fb None admin] RBAC: Authorizing `identity:get_user(user_id=4444aa1a4685411fad68fe8a8f2d8f11)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:20.965580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316856c0-7b36-49f3-84c2-a5808dc974fb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:20.967437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:20 2022] GET /identity/v3/users/4444aa1a4685411fad68fe8a8f2d8f11 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:20.972515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316856c0-7b36-49f3-84c2-a5808dc974fb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:20.986609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:21.031377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baf362-8ae5-45f8-9ba4-1d0416be3105 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:21.065295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755716b-496e-419f-a232-c553edf663f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:21.066163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755716b-496e-419f-a232-c553edf663f1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:21.066457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755716b-496e-419f-a232-c553edf663f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:21.067315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755716b-496e-419f-a232-c553edf663f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:21.069751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316856c0-7b36-49f3-84c2-a5808dc974fb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:21.085532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755716b-496e-419f-a232-c553edf663f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:21.087011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755716b-496e-419f-a232-c553edf663f1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:21.102842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b301eb-3c9a-4e79-a8df-fcf1215ac99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:21.103478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b301eb-3c9a-4e79-a8df-fcf1215ac99f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:21.103756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b301eb-3c9a-4e79-a8df-fcf1215ac99f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:21.104026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b301eb-3c9a-4e79-a8df-fcf1215ac99f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f755716b-496e-419f-a232-c553edf663f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:23:21.105016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:21.105865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:23:21.106706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:23:21.106706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:23:21.106706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:21.106871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b301eb-3c9a-4e79-a8df-fcf1215ac99f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:21.106941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:23:21.107009 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b301eb-3c9a-4e79-a8df-fcf1215ac99f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:21.108217 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:21.112018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755716b-496e-419f-a232-c553edf663f1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:21.114086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b301eb-3c9a-4e79-a8df-fcf1215ac99f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:21.172025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:21.195599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:21.213459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b301eb-3c9a-4e79-a8df-fcf1215ac99f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:21.215681 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755716b-496e-419f-a232-c553edf663f1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:21.246987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fab644-3c55-4673-a8d1-15a511995fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:21.247592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fab644-3c55-4673-a8d1-15a511995fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:21.247868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fab644-3c55-4673-a8d1-15a511995fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:21.248139 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fab644-3c55-4673-a8d1-15a511995fd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:21.248382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2acc8-72e4-4e50-a9b7-53ea37710360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:21.248853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fab644-3c55-4673-a8d1-15a511995fd4 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.domain.id=default, user.id=4444aa1a4685411fad68fe8a8f2d8f11) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:21.248986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2acc8-72e4-4e50-a9b7-53ea37710360 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:21.249161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fab644-3c55-4673-a8d1-15a511995fd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:21.249272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2acc8-72e4-4e50-a9b7-53ea37710360 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:21.249547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2acc8-72e4-4e50-a9b7-53ea37710360 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:21.250216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2acc8-72e4-4e50-a9b7-53ea37710360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:21.250525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2acc8-72e4-4e50-a9b7-53ea37710360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:21.250616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fab644-3c55-4673-a8d1-15a511995fd4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:21.251899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2acc8-72e4-4e50-a9b7-53ea37710360 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:21.257617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Tue Oct 4 05:23:21 2022] GET /identity/v3/role_assignments?user.id=4444aa1a4685411fad68fe8a8f2d8f11&scope.domain.id=default&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:21.272641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:23.706780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed3d5-5387-4d77-8043-0538a01235d8 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:23.707887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed3d5-5387-4d77-8043-0538a01235d8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:23.707962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed3d5-5387-4d77-8043-0538a01235d8 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:23.709322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:23.715446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877d4ed-3b14-4bc6-ab61-a1fd787f6417 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:23.715948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877d4ed-3b14-4bc6-ab61-a1fd787f6417 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:23.716019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877d4ed-3b14-4bc6-ab61-a1fd787f6417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:23.740497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3877d4ed-3b14-4bc6-ab61-a1fd787f6417 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:23.741723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3877d4ed-3b14-4bc6-ab61-a1fd787f6417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:23.840629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:23.882424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c629747-ead9-4e69-98e1-72f89c27052a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:23.882707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c629747-ead9-4e69-98e1-72f89c27052a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:23.885061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c629747-ead9-4e69-98e1-72f89c27052a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:23.885061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:23.895643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6941e-3043-47a5-ac0c-8fef221c9294 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:23.895930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6941e-3043-47a5-ac0c-8fef221c9294 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:23.896204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6941e-3043-47a5-ac0c-8fef221c9294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:23.904640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef92256-11f7-40d6-90b1-cb9c23be1ff2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:23.904919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef92256-11f7-40d6-90b1-cb9c23be1ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:23.905205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef92256-11f7-40d6-90b1-cb9c23be1ff2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:23.906617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 14/25] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:23:23.912861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fb8fc-b707-4415-9384-ba170b9ade57 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:23.913152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fb8fc-b707-4415-9384-ba170b9ade57 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:23.913417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fb8fc-b707-4415-9384-ba170b9ade57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:23.939223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-65f6941e-3043-47a5-ac0c-8fef221c9294 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:23.940437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-65f6941e-3043-47a5-ac0c-8fef221c9294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:23.947157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0f8fb8fc-b707-4415-9384-ba170b9ade57 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:23.948371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0f8fb8fc-b707-4415-9384-ba170b9ade57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:24.032389 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:24.096758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 12/26] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:24.104428 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 15/27] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:23:24.246116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6941e-3043-47a5-ac0c-8fef221c9294 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:24.370585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6941e-3043-47a5-ac0c-8fef221c9294 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:24.423011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fced6a1-97b2-4fc0-9918-15f4a6901597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:24.423655 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fced6a1-97b2-4fc0-9918-15f4a6901597 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:24.423933 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fced6a1-97b2-4fc0-9918-15f4a6901597 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:24.424205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fced6a1-97b2-4fc0-9918-15f4a6901597 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:24.442166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fced6a1-97b2-4fc0-9918-15f4a6901597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1fced6a1-97b2-4fc0-9918-15f4a6901597 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:24.448897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:24.449700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:24.450490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 13/28] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:23:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:24.459236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fb8fc-b707-4415-9384-ba170b9ade57 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:24.471480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fced6a1-97b2-4fc0-9918-15f4a6901597 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:24.536156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:24.630572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:24.675072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fb8fc-b707-4415-9384-ba170b9ade57 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:24.682643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fced6a1-97b2-4fc0-9918-15f4a6901597 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:24.713231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee018480-ec03-4adc-8ee6-1229c81be68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:24.714552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee018480-ec03-4adc-8ee6-1229c81be68b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:24.715025 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee018480-ec03-4adc-8ee6-1229c81be68b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:24.715535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee018480-ec03-4adc-8ee6-1229c81be68b None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:24.716498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee018480-ec03-4adc-8ee6-1229c81be68b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:24.716998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee018480-ec03-4adc-8ee6-1229c81be68b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:24.718813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee018480-ec03-4adc-8ee6-1229c81be68b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:24.720914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa20211-43fd-42d7-94f8-92b29b9c8ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:24.721534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa20211-43fd-42d7-94f8-92b29b9c8ae8 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:24.721814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa20211-43fd-42d7-94f8-92b29b9c8ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:24.722088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa20211-43fd-42d7-94f8-92b29b9c8ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:24.723517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa20211-43fd-42d7-94f8-92b29b9c8ae8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:24.724720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa20211-43fd-42d7-94f8-92b29b9c8ae8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:24.730345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 16/29] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:24.731491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee018480-ec03-4adc-8ee6-1229c81be68b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:24.844218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee018480-ec03-4adc-8ee6-1229c81be68b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:24.882710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdedfb-59e8-474c-9894-ac9601282498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:24.883670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdedfb-59e8-474c-9894-ac9601282498 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:24.884148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdedfb-59e8-474c-9894-ac9601282498 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:24.884617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdedfb-59e8-474c-9894-ac9601282498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:24.888439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:24.899038 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdedfb-59e8-474c-9894-ac9601282498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:24.902717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdedfb-59e8-474c-9894-ac9601282498 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7ebdedfb-59e8-474c-9894-ac9601282498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:23:24.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:24.919275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:23:24.920099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:23:24.920099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:23:24.920099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:24.921844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 17/30] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:24.923664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fd6ff-ac3a-4ec5-9992-797964377a9b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:24.924180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fd6ff-ac3a-4ec5-9992-797964377a9b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:24.924667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fd6ff-ac3a-4ec5-9992-797964377a9b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:24.931295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 18/31] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:24.932271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fd6ff-ac3a-4ec5-9992-797964377a9b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:24.966222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 14/32] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Oct 4 05:23:24 2022] POST /identity/v3/roles => generated 219 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:23:24.967695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa20211-43fd-42d7-94f8-92b29b9c8ae8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:25.102642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fd6ff-ac3a-4ec5-9992-797964377a9b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:25.173486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:25.183517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa20211-43fd-42d7-94f8-92b29b9c8ae8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:25.195007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625376dc-754d-4a3b-90ad-9e2a11ce6058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:25.195674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625376dc-754d-4a3b-90ad-9e2a11ce6058 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:25.195951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625376dc-754d-4a3b-90ad-9e2a11ce6058 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:25.196225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625376dc-754d-4a3b-90ad-9e2a11ce6058 None admin] PATH_INFO: `/v3/users/4444aa1a4685411fad68fe8a8f2d8f11` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:25.197465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625376dc-754d-4a3b-90ad-9e2a11ce6058 None admin] RBAC: Authorizing `identity:get_user(user_id=4444aa1a4685411fad68fe8a8f2d8f11)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:25.198935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625376dc-754d-4a3b-90ad-9e2a11ce6058 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:25.200517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 19/33] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:24 2022] GET /identity/v3/users/4444aa1a4685411fad68fe8a8f2d8f11 => generated 247 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:25.201749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec521f-da70-4208-9bb2-1862edd70862 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:25.202029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec521f-da70-4208-9bb2-1862edd70862 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:25.202328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec521f-da70-4208-9bb2-1862edd70862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:25.230065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-edec521f-da70-4208-9bb2-1862edd70862 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:25.231303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-edec521f-da70-4208-9bb2-1862edd70862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:25.235411 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444454b9-2997-4a59-a6af-982edf363444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:25.236047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444454b9-2997-4a59-a6af-982edf363444 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:25.236330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444454b9-2997-4a59-a6af-982edf363444 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:25.236606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444454b9-2997-4a59-a6af-982edf363444 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:25.237263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444454b9-2997-4a59-a6af-982edf363444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:25.237571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444454b9-2997-4a59-a6af-982edf363444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:25.240658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444454b9-2997-4a59-a6af-982edf363444 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:25.257057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 15/34] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:25.257893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444454b9-2997-4a59-a6af-982edf363444 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:25.311004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:25.330731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 20/35] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:25.331332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec521f-da70-4208-9bb2-1862edd70862 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:25.418628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444454b9-2997-4a59-a6af-982edf363444 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:25.449719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec521f-da70-4208-9bb2-1862edd70862 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:25.465388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40374e8c-d01b-498f-b1fd-038f6a5d5e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:25.466380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40374e8c-d01b-498f-b1fd-038f6a5d5e61 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:25.466912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40374e8c-d01b-498f-b1fd-038f6a5d5e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:25.467495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40374e8c-d01b-498f-b1fd-038f6a5d5e61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:25.469173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40374e8c-d01b-498f-b1fd-038f6a5d5e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:25.471130 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40374e8c-d01b-498f-b1fd-038f6a5d5e61 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:25.473552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 16/36] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:25.475921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2614786-13b9-45bb-afa8-6b9ba11d5970 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:25.476764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2614786-13b9-45bb-afa8-6b9ba11d5970 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:25.477565 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2614786-13b9-45bb-afa8-6b9ba11d5970 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:25.479401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 17/37] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:25.480392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2614786-13b9-45bb-afa8-6b9ba11d5970 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:25.495196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e711364-1c84-4c58-a88a-87ac621bde53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:25.496083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e711364-1c84-4c58-a88a-87ac621bde53 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:25.496367 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e711364-1c84-4c58-a88a-87ac621bde53 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:25.496641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e711364-1c84-4c58-a88a-87ac621bde53 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:25.497353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e711364-1c84-4c58-a88a-87ac621bde53 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:25.499519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e711364-1c84-4c58-a88a-87ac621bde53 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:25.510669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:25.596617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2614786-13b9-45bb-afa8-6b9ba11d5970 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:25.635983 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6b4eb-3d04-47fa-9819-05bce12bd81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:25.636653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6b4eb-3d04-47fa-9819-05bce12bd81c None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:25.636932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6b4eb-3d04-47fa-9819-05bce12bd81c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:25.637203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6b4eb-3d04-47fa-9819-05bce12bd81c None admin] PATH_INFO: `/v3/domains/default/users/4444aa1a4685411fad68fe8a8f2d8f11/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:25.639548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6b4eb-3d04-47fa-9819-05bce12bd81c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4444aa1a4685411fad68fe8a8f2d8f11, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:25.640962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6b4eb-3d04-47fa-9819-05bce12bd81c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:25.717974 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 18/38] 10.222.0.26 () {58 vars in 1361 bytes} [Tue Oct 4 05:23:25 2022] PUT /identity/v3/domains/default/users/4444aa1a4685411fad68fe8a8f2d8f11/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:23:25.719251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a4c99-f4c5-4fed-8b11-903fc168f13c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:25.719549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a4c99-f4c5-4fed-8b11-903fc168f13c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:25.719836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a4c99-f4c5-4fed-8b11-903fc168f13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:25.738462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 21/39] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:23:25 2022] POST /identity/v3/projects => generated 301 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:23:25.738462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e711364-1c84-4c58-a88a-87ac621bde53 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:25.740634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f31a4c99-f4c5-4fed-8b11-903fc168f13c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:25.742511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f31a4c99-f4c5-4fed-8b11-903fc168f13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:25.850547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 19/40] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:25.855833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087c193-41cd-4d29-9de2-633b84c76e93 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:25.856138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087c193-41cd-4d29-9de2-633b84c76e93 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:25.856488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087c193-41cd-4d29-9de2-633b84c76e93 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:25.858305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 20/41] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:25.866193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103a00-23bc-4272-951a-d3637476bfd1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:25.866491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103a00-23bc-4272-951a-d3637476bfd1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:25.866770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103a00-23bc-4272-951a-d3637476bfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:25.908313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7d103a00-23bc-4272-951a-d3637476bfd1 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:25.909580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7d103a00-23bc-4272-951a-d3637476bfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:25.926564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:25.929250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:26.005465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 21/42] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:26.014480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e711364-1c84-4c58-a88a-87ac621bde53 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:26.056992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5a1c2-f31e-41a8-9169-092f1c9200b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:26.057667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5a1c2-f31e-41a8-9169-092f1c9200b8 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:26.057952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5a1c2-f31e-41a8-9169-092f1c9200b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:26.058254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5a1c2-f31e-41a8-9169-092f1c9200b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:26.059047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5a1c2-f31e-41a8-9169-092f1c9200b8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:26.060583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5a1c2-f31e-41a8-9169-092f1c9200b8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:26.151839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 22/43] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Oct 4 05:23:25 2022] POST /identity/v3/roles => generated 225 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:23:26.184925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d103a00-23bc-4272-951a-d3637476bfd1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:26.303478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d103a00-23bc-4272-951a-d3637476bfd1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:26.309764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5a1c2-f31e-41a8-9169-092f1c9200b8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:26.341848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d091-c2d1-4e56-b89d-71304ee53ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:26.342487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d091-c2d1-4e56-b89d-71304ee53ee7 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:26.342766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d091-c2d1-4e56-b89d-71304ee53ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:26.343040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d091-c2d1-4e56-b89d-71304ee53ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:26.343737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55d091-c2d1-4e56-b89d-71304ee53ee7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:26.345730 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55d091-c2d1-4e56-b89d-71304ee53ee7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:26.427042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:26.430928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5a1c2-f31e-41a8-9169-092f1c9200b8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:26.436093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 22/44] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Oct 4 05:23:26 2022] POST /identity/v3/roles => generated 223 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:23:26.467468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55cb59-1130-4712-974a-cada36b1fdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:26.468140 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55cb59-1130-4712-974a-cada36b1fdf7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:26.468419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55cb59-1130-4712-974a-cada36b1fdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:26.468697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55cb59-1130-4712-974a-cada36b1fdf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:26.469965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55cb59-1130-4712-974a-cada36b1fdf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:26.471505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55cb59-1130-4712-974a-cada36b1fdf7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:26.473385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 23/45] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:26.479069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d091-c2d1-4e56-b89d-71304ee53ee7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:26.483041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:26.590168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d091-c2d1-4e56-b89d-71304ee53ee7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:26.630717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6971f5-d701-46e5-ad92-3aa9c64bd5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:26.631343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6971f5-d701-46e5-ad92-3aa9c64bd5e3 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:26.631627 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6971f5-d701-46e5-ad92-3aa9c64bd5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:26.631909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6971f5-d701-46e5-ad92-3aa9c64bd5e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:26.632565 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6971f5-d701-46e5-ad92-3aa9c64bd5e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:26.634027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6971f5-d701-46e5-ad92-3aa9c64bd5e3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:26.720032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 23/46] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:23:26 2022] POST /identity/v3/projects => generated 312 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:23:27.978479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02d1f9-7aa9-490b-8c3a-0e07b23684c8 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:27.978973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02d1f9-7aa9-490b-8c3a-0e07b23684c8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:27.979074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02d1f9-7aa9-490b-8c3a-0e07b23684c8 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:27.980350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 24/47] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:27.985729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbabdb0-b39f-484e-88c0-3a82c0c7adef None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:27.986116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbabdb0-b39f-484e-88c0-3a82c0c7adef None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:27.986322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbabdb0-b39f-484e-88c0-3a82c0c7adef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:28.107501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2dbabdb0-b39f-484e-88c0-3a82c0c7adef None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:28.108701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2dbabdb0-b39f-484e-88c0-3a82c0c7adef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:28.156537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:28.172561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 24/48] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:28.436726 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02d1f9-7aa9-490b-8c3a-0e07b23684c8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:28.536112 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02d1f9-7aa9-490b-8c3a-0e07b23684c8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:28.568298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496eceb-30aa-4728-850e-2dfaf2ef2fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:28.568918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496eceb-30aa-4728-850e-2dfaf2ef2fed None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:28.569195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496eceb-30aa-4728-850e-2dfaf2ef2fed None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:28.569465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496eceb-30aa-4728-850e-2dfaf2ef2fed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:28.579202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496eceb-30aa-4728-850e-2dfaf2ef2fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-1496eceb-30aa-4728-850e-2dfaf2ef2fed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:28.584556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:28.585403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:28.586193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 25/49] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:23:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:28.592735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbabdb0-b39f-484e-88c0-3a82c0c7adef None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:28.688080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbabdb0-b39f-484e-88c0-3a82c0c7adef None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:28.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77509e59-b657-4b55-80ee-8edb1258c690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:28.721228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77509e59-b657-4b55-80ee-8edb1258c690 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:28.721510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77509e59-b657-4b55-80ee-8edb1258c690 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:28.721784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77509e59-b657-4b55-80ee-8edb1258c690 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:28.722480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77509e59-b657-4b55-80ee-8edb1258c690 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:28.723330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77509e59-b657-4b55-80ee-8edb1258c690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:28.724687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77509e59-b657-4b55-80ee-8edb1258c690 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:28.735959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 25/50] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:28.743101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496eceb-30aa-4728-850e-2dfaf2ef2fed None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:28.772688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:28.845257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496eceb-30aa-4728-850e-2dfaf2ef2fed None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:28.877205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b69430-e549-4332-9ab8-78a24e7ed2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:28.877839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b69430-e549-4332-9ab8-78a24e7ed2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:28.878118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b69430-e549-4332-9ab8-78a24e7ed2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:28.878419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b69430-e549-4332-9ab8-78a24e7ed2a5 None admin] PATH_INFO: `/v3/users/4444aa1a4685411fad68fe8a8f2d8f11` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:28.880085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b69430-e549-4332-9ab8-78a24e7ed2a5 None admin] RBAC: Authorizing `identity:get_user(user_id=4444aa1a4685411fad68fe8a8f2d8f11)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:28.881530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b69430-e549-4332-9ab8-78a24e7ed2a5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:28.883136 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 26/51] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:28 2022] GET /identity/v3/users/4444aa1a4685411fad68fe8a8f2d8f11 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:28.885964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a377b65-b4ab-4b53-9529-1b7f7c0a6c52 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:28.886330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a377b65-b4ab-4b53-9529-1b7f7c0a6c52 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:28.886563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a377b65-b4ab-4b53-9529-1b7f7c0a6c52 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:28.888019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 26/52] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:28.890648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b69430-e549-4332-9ab8-78a24e7ed2a5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:28.893115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18216ffd-226c-4722-a43a-bf269d4258c2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:28.893401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18216ffd-226c-4722-a43a-bf269d4258c2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:28.893682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18216ffd-226c-4722-a43a-bf269d4258c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:28.912257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-18216ffd-226c-4722-a43a-bf269d4258c2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:28.913442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-18216ffd-226c-4722-a43a-bf269d4258c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:28.958745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:28.981803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 27/53] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:23:28.984504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:28.990916 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b69430-e549-4332-9ab8-78a24e7ed2a5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:29.024395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d3590-63dd-461b-9718-7ee657d17d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:29.025027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d3590-63dd-461b-9718-7ee657d17d51 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:29.025312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d3590-63dd-461b-9718-7ee657d17d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:29.025589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d3590-63dd-461b-9718-7ee657d17d51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:29.026735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d3590-63dd-461b-9718-7ee657d17d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:29.028286 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d3590-63dd-461b-9718-7ee657d17d51 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:29.029789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 27/54] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:29.035750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18216ffd-226c-4722-a43a-bf269d4258c2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:29.133330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18216ffd-226c-4722-a43a-bf269d4258c2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:29.164997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799669a9-dd65-42e0-a86f-b62b9507df03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:29.165599 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799669a9-dd65-42e0-a86f-b62b9507df03 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:29.165880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799669a9-dd65-42e0-a86f-b62b9507df03 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:29.166181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799669a9-dd65-42e0-a86f-b62b9507df03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:29.166901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799669a9-dd65-42e0-a86f-b62b9507df03 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.domain.id=default, user.id=4444aa1a4685411fad68fe8a8f2d8f11) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:29.167209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799669a9-dd65-42e0-a86f-b62b9507df03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:29.168654 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799669a9-dd65-42e0-a86f-b62b9507df03 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:29.176803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 28/55] 10.222.0.26 () {56 vars in 1322 bytes} [Tue Oct 4 05:23:29 2022] GET /identity/v3/role_assignments?user.id=4444aa1a4685411fad68fe8a8f2d8f11&scope.domain.id=default&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:29.248319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d3590-63dd-461b-9718-7ee657d17d51 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:29.352448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d3590-63dd-461b-9718-7ee657d17d51 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:29.389536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51453d59-e855-4cbc-b2eb-0b45038199b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:29.390213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51453d59-e855-4cbc-b2eb-0b45038199b7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:29.390498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51453d59-e855-4cbc-b2eb-0b45038199b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:29.390775 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51453d59-e855-4cbc-b2eb-0b45038199b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:29.391886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51453d59-e855-4cbc-b2eb-0b45038199b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:29.393326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51453d59-e855-4cbc-b2eb-0b45038199b7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:29.395169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 28/56] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:29.400584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799669a9-dd65-42e0-a86f-b62b9507df03 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:29.444639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:29.497682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799669a9-dd65-42e0-a86f-b62b9507df03 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:29.529289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf9f6d-2d63-459c-88cc-ae488d0f1de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:29.529832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf9f6d-2d63-459c-88cc-ae488d0f1de3 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:29.530111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf9f6d-2d63-459c-88cc-ae488d0f1de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:29.530418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf9f6d-2d63-459c-88cc-ae488d0f1de3 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:29.531053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf9f6d-2d63-459c-88cc-ae488d0f1de3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:29.532488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf9f6d-2d63-459c-88cc-ae488d0f1de3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:29.601600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 29/57] 10.222.0.26 () {60 vars in 1140 bytes} [Tue Oct 4 05:23:29 2022] POST /identity/v3/projects => generated 298 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:23:31.721389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f7186-1218-4e18-a7e6-8a9c434f361b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:31.721902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f7186-1218-4e18-a7e6-8a9c434f361b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:31.722005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f7186-1218-4e18-a7e6-8a9c434f361b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:31.723894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 29/58] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:31.729058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c0b52-de8d-4971-b953-39a9d5ebbc79 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:31.729435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c0b52-de8d-4971-b953-39a9d5ebbc79 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:31.729617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c0b52-de8d-4971-b953-39a9d5ebbc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:31.747003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3b0c0b52-de8d-4971-b953-39a9d5ebbc79 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:31.748194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3b0c0b52-de8d-4971-b953-39a9d5ebbc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:31.810695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 30/59] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:32.084269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f7186-1218-4e18-a7e6-8a9c434f361b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:32.114310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:32.182123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f7186-1218-4e18-a7e6-8a9c434f361b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:32.214864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fdf79-8bd1-4ee3-bb82-9a88b09bad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:32.215489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fdf79-8bd1-4ee3-bb82-9a88b09bad40 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:32.215770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fdf79-8bd1-4ee3-bb82-9a88b09bad40 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:32.216045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fdf79-8bd1-4ee3-bb82-9a88b09bad40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:32.217148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345fdf79-8bd1-4ee3-bb82-9a88b09bad40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:32.218607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345fdf79-8bd1-4ee3-bb82-9a88b09bad40 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:32.220146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 30/60] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:32.225486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c0b52-de8d-4971-b953-39a9d5ebbc79 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:32.249313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:32.321787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c0b52-de8d-4971-b953-39a9d5ebbc79 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:32.353091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b518b8-4506-4616-8da8-680921510e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:32.353692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b518b8-4506-4616-8da8-680921510e75 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:32.353969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b518b8-4506-4616-8da8-680921510e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:32.354268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b518b8-4506-4616-8da8-680921510e75 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:32.354917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b518b8-4506-4616-8da8-680921510e75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:32.356354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b518b8-4506-4616-8da8-680921510e75 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:32.443499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 31/61] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:23:32 2022] POST /identity/v3/users => generated 275 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:23:35.614026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7e31d-4600-4357-948f-a5632e7d7a83 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:35.628973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7e31d-4600-4357-948f-a5632e7d7a83 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:35.628973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7e31d-4600-4357-948f-a5632e7d7a83 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:35.628973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 31/62] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:35.629205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56e7c8-d1cd-492b-964b-e2168f317468 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:35.629205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56e7c8-d1cd-492b-964b-e2168f317468 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:35.629205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56e7c8-d1cd-492b-964b-e2168f317468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:35.646336 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dc56e7c8-d1cd-492b-964b-e2168f317468 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:35.647642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dc56e7c8-d1cd-492b-964b-e2168f317468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:35.741100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 32/63] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:35.758741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9459-3fc9-47f4-bd87-96611a7b453f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:35.759033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9459-3fc9-47f4-bd87-96611a7b453f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:35.759302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9459-3fc9-47f4-bd87-96611a7b453f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:35.760774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 32/64] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:35.766017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd82e4-3ee7-4ae8-8c9c-69878840d73e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:35.766338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd82e4-3ee7-4ae8-8c9c-69878840d73e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:35.766624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd82e4-3ee7-4ae8-8c9c-69878840d73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:35.791173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4acd82e4-3ee7-4ae8-8c9c-69878840d73e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:35.792436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4acd82e4-3ee7-4ae8-8c9c-69878840d73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:35.795539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:35.884737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 33/65] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:35.986879 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663a975-729f-4750-a56b-0b9b692edab5 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:35.987195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663a975-729f-4750-a56b-0b9b692edab5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:35.987444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663a975-729f-4750-a56b-0b9b692edab5 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:35.988820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 33/66] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:36.003383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157e422-fdba-4781-b7b6-271e06ffe104 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.003700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157e422-fdba-4781-b7b6-271e06ffe104 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.003984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157e422-fdba-4781-b7b6-271e06ffe104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.031188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9157e422-fdba-4781-b7b6-271e06ffe104 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:36.032395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9157e422-fdba-4781-b7b6-271e06ffe104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:36.075150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663a975-729f-4750-a56b-0b9b692edab5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:36.149158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 34/67] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:36.224869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157e422-fdba-4781-b7b6-271e06ffe104 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:36.232743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:36.242795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663a975-729f-4750-a56b-0b9b692edab5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:36.317358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152550c-6acf-43fd-af25-434db1769c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:36.318315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152550c-6acf-43fd-af25-434db1769c23 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.318602 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152550c-6acf-43fd-af25-434db1769c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.318880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152550c-6acf-43fd-af25-434db1769c23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.341553 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152550c-6acf-43fd-af25-434db1769c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f152550c-6acf-43fd-af25-434db1769c23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:36.346556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:36.347451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:36.348215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 34/68] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:36.356055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152550c-6acf-43fd-af25-434db1769c23 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:36.370503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:36.401596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157e422-fdba-4781-b7b6-271e06ffe104 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:36.442554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea798563-98dc-4ae1-affb-b6efcaecbc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:36.443846 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea798563-98dc-4ae1-affb-b6efcaecbc17 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.444356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea798563-98dc-4ae1-affb-b6efcaecbc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.444820 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea798563-98dc-4ae1-affb-b6efcaecbc17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.459889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea798563-98dc-4ae1-affb-b6efcaecbc17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ea798563-98dc-4ae1-affb-b6efcaecbc17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:36.466455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:36.467388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:36.468132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:36.470380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 35/69] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:23:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:23:36.472497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9b5d9-87ce-44a6-8a72-4a5b6fd77fce None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.472979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9b5d9-87ce-44a6-8a72-4a5b6fd77fce None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.473443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9b5d9-87ce-44a6-8a72-4a5b6fd77fce None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.475167 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 36/70] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:36.477208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9b5d9-87ce-44a6-8a72-4a5b6fd77fce None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:36.515345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152550c-6acf-43fd-af25-434db1769c23 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:36.549938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de194291-8b26-4646-9dd8-f04d743b0559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:36.550622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de194291-8b26-4646-9dd8-f04d743b0559 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.550900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de194291-8b26-4646-9dd8-f04d743b0559 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.551173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de194291-8b26-4646-9dd8-f04d743b0559 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.551837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de194291-8b26-4646-9dd8-f04d743b0559 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:36.552138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de194291-8b26-4646-9dd8-f04d743b0559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:36.553510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de194291-8b26-4646-9dd8-f04d743b0559 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:36.561880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 35/71] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:36.563099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0529b9c-b159-4e2b-9e3c-2b79edf4f71d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.563376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0529b9c-b159-4e2b-9e3c-2b79edf4f71d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.563659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0529b9c-b159-4e2b-9e3c-2b79edf4f71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.582629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b0529b9c-b159-4e2b-9e3c-2b79edf4f71d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:36.584153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b0529b9c-b159-4e2b-9e3c-2b79edf4f71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:36.587152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9b5d9-87ce-44a6-8a72-4a5b6fd77fce None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:36.598048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:36.615810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:36.621559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b4c13-93d9-401b-968a-1431763f8e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:36.622198 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b4c13-93d9-401b-968a-1431763f8e5e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.622488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b4c13-93d9-401b-968a-1431763f8e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.622769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b4c13-93d9-401b-968a-1431763f8e5e None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.623442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b4c13-93d9-401b-968a-1431763f8e5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:36.623759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b4c13-93d9-401b-968a-1431763f8e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:36.625216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b4c13-93d9-401b-968a-1431763f8e5e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:36.633712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 37/72] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:36.634542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b4c13-93d9-401b-968a-1431763f8e5e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:36.651724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 36/73] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:36.652915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bcc69-73fb-4ddc-9e80-7a393b2bdced None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.653211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bcc69-73fb-4ddc-9e80-7a393b2bdced None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.653486 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472bcc69-73fb-4ddc-9e80-7a393b2bdced None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.654745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 37/74] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:36.659042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bcc69-73fb-4ddc-9e80-7a393b2bdced None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:36.748845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b4c13-93d9-401b-968a-1431763f8e5e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:36.760390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472bcc69-73fb-4ddc-9e80-7a393b2bdced None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:36.789182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c86d51-1856-4cfb-91b5-af944b1c15f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:36.789899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c86d51-1856-4cfb-91b5-af944b1c15f0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.790270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c86d51-1856-4cfb-91b5-af944b1c15f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.790618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c86d51-1856-4cfb-91b5-af944b1c15f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.793984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e17e3-1330-47b8-a133-864914c04c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:36.794652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e17e3-1330-47b8-a133-864914c04c58 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.794932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e17e3-1330-47b8-a133-864914c04c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.795215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e17e3-1330-47b8-a133-864914c04c58 None admin] PATH_INFO: `/v3/users/f0f33c461e67433593c4ca55f81f4e7e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.802070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c86d51-1856-4cfb-91b5-af944b1c15f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-76c86d51-1856-4cfb-91b5-af944b1c15f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:36.807391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:36.808247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:36.813978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 38/75] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:23:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:36.814764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e17e3-1330-47b8-a133-864914c04c58 None admin] RBAC: Authorizing `identity:get_user(user_id=f0f33c461e67433593c4ca55f81f4e7e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:36.814833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c86d51-1856-4cfb-91b5-af944b1c15f0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:36.814895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e17e3-1330-47b8-a133-864914c04c58 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:36.814966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 38/76] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:36 2022] GET /identity/v3/users/f0f33c461e67433593c4ca55f81f4e7e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:36.821466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c268a0-f750-4595-a602-d8e504179688 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.821769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c268a0-f750-4595-a602-d8e504179688 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.822061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c268a0-f750-4595-a602-d8e504179688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.839299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b4c268a0-f750-4595-a602-d8e504179688 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:36.840496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b4c268a0-f750-4595-a602-d8e504179688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:36.842483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:36.868274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:36.910727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 39/77] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:36.911861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c268a0-f750-4595-a602-d8e504179688 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:36.928779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c86d51-1856-4cfb-91b5-af944b1c15f0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:36.963837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b361461-8167-472b-8b4a-ebbed989e544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:36.964764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b361461-8167-472b-8b4a-ebbed989e544 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:36.965244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b361461-8167-472b-8b4a-ebbed989e544 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:36.965716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b361461-8167-472b-8b4a-ebbed989e544 None admin] PATH_INFO: `/v3/users/4444aa1a4685411fad68fe8a8f2d8f11` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:36.967186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b361461-8167-472b-8b4a-ebbed989e544 None admin] RBAC: Authorizing `identity:get_user(user_id=4444aa1a4685411fad68fe8a8f2d8f11)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:36.968796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b361461-8167-472b-8b4a-ebbed989e544 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:36.970612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 39/78] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:36 2022] GET /identity/v3/users/4444aa1a4685411fad68fe8a8f2d8f11 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:36.971567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b361461-8167-472b-8b4a-ebbed989e544 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:37.028174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c268a0-f750-4595-a602-d8e504179688 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:37.063384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec9e2d-f74e-42f9-9248-657bd33663a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:37.063994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec9e2d-f74e-42f9-9248-657bd33663a8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:37.064275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec9e2d-f74e-42f9-9248-657bd33663a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:37.064549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec9e2d-f74e-42f9-9248-657bd33663a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:37.065229 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec9e2d-f74e-42f9-9248-657bd33663a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:37.065512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec9e2d-f74e-42f9-9248-657bd33663a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:37.067465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec9e2d-f74e-42f9-9248-657bd33663a8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:37.072654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:37.076764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 40/79] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:23:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:37.077381 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec9e2d-f74e-42f9-9248-657bd33663a8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:37.081747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b361461-8167-472b-8b4a-ebbed989e544 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:37.103023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:37.116585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d92dc-cfa0-4166-b06c-dbe0812a6a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:37.117570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d92dc-cfa0-4166-b06c-dbe0812a6a9b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:37.117856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d92dc-cfa0-4166-b06c-dbe0812a6a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:37.118153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d92dc-cfa0-4166-b06c-dbe0812a6a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:37.177358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec9e2d-f74e-42f9-9248-657bd33663a8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:37.188562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d92dc-cfa0-4166-b06c-dbe0812a6a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:37.190113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d92dc-cfa0-4166-b06c-dbe0812a6a9b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:37.212218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222469f6-d057-4c2a-b76b-41c8232f28ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:37.212850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222469f6-d057-4c2a-b76b-41c8232f28ed None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:37.213128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222469f6-d057-4c2a-b76b-41c8232f28ed None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:37.213400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222469f6-d057-4c2a-b76b-41c8232f28ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:37.214535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222469f6-d057-4c2a-b76b-41c8232f28ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:37.215957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222469f6-d057-4c2a-b76b-41c8232f28ed None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:37.217716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 41/80] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:37.218300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222469f6-d057-4c2a-b76b-41c8232f28ed None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:37.284264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:37.287632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 40/81] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:23:37.292421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d92dc-cfa0-4166-b06c-dbe0812a6a9b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:37.310082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:37.318498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222469f6-d057-4c2a-b76b-41c8232f28ed None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:37.352631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f426e03-62d1-437c-84fd-5e1a31a82e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:37.353236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f426e03-62d1-437c-84fd-5e1a31a82e48 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:37.353518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f426e03-62d1-437c-84fd-5e1a31a82e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:37.353794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f426e03-62d1-437c-84fd-5e1a31a82e48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:37.387057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d92dc-cfa0-4166-b06c-dbe0812a6a9b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:37.419728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f426e03-62d1-437c-84fd-5e1a31a82e48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:37.419875 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb114e77-8718-469e-8113-42bdc3112e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:37.420385 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb114e77-8718-469e-8113-42bdc3112e5e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:37.420664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb114e77-8718-469e-8113-42bdc3112e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:37.420936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb114e77-8718-469e-8113-42bdc3112e5e None admin] PATH_INFO: `/v3/users/f0f33c461e67433593c4ca55f81f4e7e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:37.421216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f426e03-62d1-437c-84fd-5e1a31a82e48 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:37.422065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb114e77-8718-469e-8113-42bdc3112e5e None admin] RBAC: Authorizing `identity:get_user(user_id=f0f33c461e67433593c4ca55f81f4e7e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:37.423496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb114e77-8718-469e-8113-42bdc3112e5e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:37.425050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 41/82] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:37 2022] GET /identity/v3/users/f0f33c461e67433593c4ca55f81f4e7e => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:37.427037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb114e77-8718-469e-8113-42bdc3112e5e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:37.485036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:37.513206 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:37.519080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 42/83] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:23:37.519981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f426e03-62d1-437c-84fd-5e1a31a82e48 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:37.529361 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb114e77-8718-469e-8113-42bdc3112e5e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:37.564473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc10e49-7528-430e-834b-75d6b2788b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:37.565112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10e49-7528-430e-834b-75d6b2788b9c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:37.565396 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10e49-7528-430e-834b-75d6b2788b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:37.565670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10e49-7528-430e-834b-75d6b2788b9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:37.576458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc10e49-7528-430e-834b-75d6b2788b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ffc10e49-7528-430e-834b-75d6b2788b9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:37.581357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:37.582406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:37.583354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 42/84] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:37.587153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc10e49-7528-430e-834b-75d6b2788b9c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:37.619649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f426e03-62d1-437c-84fd-5e1a31a82e48 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:37.652225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373e101-3511-4ee8-949b-9bfbe2b6dfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:37.652936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373e101-3511-4ee8-949b-9bfbe2b6dfa1 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:37.653118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373e101-3511-4ee8-949b-9bfbe2b6dfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:37.653395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373e101-3511-4ee8-949b-9bfbe2b6dfa1 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:37.654057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373e101-3511-4ee8-949b-9bfbe2b6dfa1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:37.655543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373e101-3511-4ee8-949b-9bfbe2b6dfa1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:37.686035 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc10e49-7528-430e-834b-75d6b2788b9c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:37.719885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bc947-1f93-409b-aaa8-de8aeb913774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:37.720545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bc947-1f93-409b-aaa8-de8aeb913774 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:37.720826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bc947-1f93-409b-aaa8-de8aeb913774 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:37.721101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bc947-1f93-409b-aaa8-de8aeb913774 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:37.722325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bc947-1f93-409b-aaa8-de8aeb913774 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:37.723776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bc947-1f93-409b-aaa8-de8aeb913774 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:37.725436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 43/85] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:37 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:37.726220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bc947-1f93-409b-aaa8-de8aeb913774 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:37.744031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:37.749074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 43/86] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:23:37 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:23:37.758571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373e101-3511-4ee8-949b-9bfbe2b6dfa1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:37.782302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:37.848276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bc947-1f93-409b-aaa8-de8aeb913774 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:37.868266 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373e101-3511-4ee8-949b-9bfbe2b6dfa1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:37.888558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264170a1-9ec3-4146-9540-c79bb2bce7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:37.889420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264170a1-9ec3-4146-9540-c79bb2bce7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:37.889841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264170a1-9ec3-4146-9540-c79bb2bce7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:37.890868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264170a1-9ec3-4146-9540-c79bb2bce7f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:37.911103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffc220-4c94-4e79-abd6-6f6cad92658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:37.911773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffc220-4c94-4e79-abd6-6f6cad92658f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:37.912072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffc220-4c94-4e79-abd6-6f6cad92658f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:37.912327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffc220-4c94-4e79-abd6-6f6cad92658f None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:37.913601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffc220-4c94-4e79-abd6-6f6cad92658f None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:37.915071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffc220-4c94-4e79-abd6-6f6cad92658f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:37.916941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 44/87] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:37 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:37.922453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffc220-4c94-4e79-abd6-6f6cad92658f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:37.975296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264170a1-9ec3-4146-9540-c79bb2bce7f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:37.977136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264170a1-9ec3-4146-9540-c79bb2bce7f0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:37.995489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:38.050071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:38.059998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffc220-4c94-4e79-abd6-6f6cad92658f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:38.106036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 44/88] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:23:38.106837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264170a1-9ec3-4146-9540-c79bb2bce7f0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:38.111627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9484063-2b61-4fd6-8b08-403b2dcef5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:38.112266 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9484063-2b61-4fd6-8b08-403b2dcef5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:38.112547 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9484063-2b61-4fd6-8b08-403b2dcef5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:38.113633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9484063-2b61-4fd6-8b08-403b2dcef5f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:38.114115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9484063-2b61-4fd6-8b08-403b2dcef5f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:38.114472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9484063-2b61-4fd6-8b08-403b2dcef5f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:38.115860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9484063-2b61-4fd6-8b08-403b2dcef5f7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:38.124591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 45/89] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:38.125408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9484063-2b61-4fd6-8b08-403b2dcef5f7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:38.225710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264170a1-9ec3-4146-9540-c79bb2bce7f0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:38.234629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9484063-2b61-4fd6-8b08-403b2dcef5f7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:38.247906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:38.274584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f132f-5494-4b38-afee-235439967dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:38.275553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f132f-5494-4b38-afee-235439967dfe None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:38.276059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f132f-5494-4b38-afee-235439967dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:38.276535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f132f-5494-4b38-afee-235439967dfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:38.276635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939e40-ba9b-46fa-b518-7f298dcc104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:38.277155 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939e40-ba9b-46fa-b518-7f298dcc104c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:38.277436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939e40-ba9b-46fa-b518-7f298dcc104c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:38.277712 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939e40-ba9b-46fa-b518-7f298dcc104c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:38.278225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f132f-5494-4b38-afee-235439967dfe None admin] RBAC: Adding query filter params (role.id=1e04fdc0cc334fd7a733a1490a194e37, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user.id=f0f33c461e67433593c4ca55f81f4e7e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:38.278477 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2939e40-ba9b-46fa-b518-7f298dcc104c None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user.id=4444aa1a4685411fad68fe8a8f2d8f11) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:38.278787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2939e40-ba9b-46fa-b518-7f298dcc104c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:38.279127 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f132f-5494-4b38-afee-235439967dfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:38.280302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2939e40-ba9b-46fa-b518-7f298dcc104c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:38.282511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f132f-5494-4b38-afee-235439967dfe None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:38.287049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 46/90] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:38 2022] GET /identity/v3/role_assignments?user.id=4444aa1a4685411fad68fe8a8f2d8f11&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:38.287968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939e40-ba9b-46fa-b518-7f298dcc104c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:38.291188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 45/91] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:38 2022] GET /identity/v3/role_assignments?user.id=f0f33c461e67433593c4ca55f81f4e7e&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:38.292371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f132f-5494-4b38-afee-235439967dfe None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:38.360027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:38.425903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f132f-5494-4b38-afee-235439967dfe None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:38.488037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603918b4-3192-4bcc-b21c-616b7643a5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:38.489046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603918b4-3192-4bcc-b21c-616b7643a5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:38.489560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603918b4-3192-4bcc-b21c-616b7643a5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:38.490049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603918b4-3192-4bcc-b21c-616b7643a5c6 None admin] PATH_INFO: `/v3/users/f0f33c461e67433593c4ca55f81f4e7e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:38.491914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603918b4-3192-4bcc-b21c-616b7643a5c6 None admin] RBAC: Authorizing `identity:get_user(user_id=f0f33c461e67433593c4ca55f81f4e7e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:38.495843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603918b4-3192-4bcc-b21c-616b7643a5c6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:38.500359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 46/92] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:38 2022] GET /identity/v3/users/f0f33c461e67433593c4ca55f81f4e7e => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:38.506704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603918b4-3192-4bcc-b21c-616b7643a5c6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:38.563143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939e40-ba9b-46fa-b518-7f298dcc104c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:38.605262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c711fb-ea8c-4fe9-a894-437993f9f32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:38.607912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:38.608964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c711fb-ea8c-4fe9-a894-437993f9f32a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:38.610363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c711fb-ea8c-4fe9-a894-437993f9f32a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:38.613634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c711fb-ea8c-4fe9-a894-437993f9f32a None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:38.615150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c711fb-ea8c-4fe9-a894-437993f9f32a None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:38.617975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c711fb-ea8c-4fe9-a894-437993f9f32a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:38.620143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 47/93] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:38 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:38.626103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c711fb-ea8c-4fe9-a894-437993f9f32a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:38.737181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603918b4-3192-4bcc-b21c-616b7643a5c6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:38.749935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:38.770565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c711fb-ea8c-4fe9-a894-437993f9f32a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:38.793985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681d1ec-818f-4f76-a327-77149d5c0777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:38.794655 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681d1ec-818f-4f76-a327-77149d5c0777 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:38.794939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681d1ec-818f-4f76-a327-77149d5c0777 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:38.795267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681d1ec-818f-4f76-a327-77149d5c0777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:38.806195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103de140-b7d3-4981-96dc-3c7fd38887ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:38.806831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103de140-b7d3-4981-96dc-3c7fd38887ae None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:38.808120 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103de140-b7d3-4981-96dc-3c7fd38887ae None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:38.808414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103de140-b7d3-4981-96dc-3c7fd38887ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:38.809250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103de140-b7d3-4981-96dc-3c7fd38887ae None admin] RBAC: Adding query filter params (role.id=cae161e9c1b64656924448dd784ed4d1, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user.id=f0f33c461e67433593c4ca55f81f4e7e) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:38.812153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103de140-b7d3-4981-96dc-3c7fd38887ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:38.813633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103de140-b7d3-4981-96dc-3c7fd38887ae None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:38.824065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 48/94] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:38 2022] GET /identity/v3/role_assignments?user.id=f0f33c461e67433593c4ca55f81f4e7e&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=cae161e9c1b64656924448dd784ed4d1 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:38.885628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681d1ec-818f-4f76-a327-77149d5c0777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:38.887195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681d1ec-818f-4f76-a327-77149d5c0777 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:38.963782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:39.027813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 47/95] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:23:39.033757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103de140-b7d3-4981-96dc-3c7fd38887ae None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:39.148876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103de140-b7d3-4981-96dc-3c7fd38887ae None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:39.192208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc783c-77a6-4f0a-9b9d-a9da2d37ce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:39.193207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc783c-77a6-4f0a-9b9d-a9da2d37ce4c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:39.193494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc783c-77a6-4f0a-9b9d-a9da2d37ce4c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:39.193775 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc783c-77a6-4f0a-9b9d-a9da2d37ce4c None admin] PATH_INFO: `/v3/projects/c13bf11626c248379284b9d7e67527bc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:39.195243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc783c-77a6-4f0a-9b9d-a9da2d37ce4c None admin] RBAC: Authorizing `identity:get_project(project_id=c13bf11626c248379284b9d7e67527bc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:39.196765 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc783c-77a6-4f0a-9b9d-a9da2d37ce4c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:39.199108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 49/96] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:39 2022] GET /identity/v3/projects/c13bf11626c248379284b9d7e67527bc => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:39.205086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681d1ec-818f-4f76-a327-77149d5c0777 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:39.326229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681d1ec-818f-4f76-a327-77149d5c0777 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:39.371108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48d983-d24c-4e56-93c1-b838515b037b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:39.371784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48d983-d24c-4e56-93c1-b838515b037b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:39.372076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48d983-d24c-4e56-93c1-b838515b037b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:39.372358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48d983-d24c-4e56-93c1-b838515b037b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:39.373092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48d983-d24c-4e56-93c1-b838515b037b None admin] RBAC: Adding query filter params (role.id=1e04fdc0cc334fd7a733a1490a194e37, scope.project.id=c13bf11626c248379284b9d7e67527bc, user.id=f0f33c461e67433593c4ca55f81f4e7e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:39.373405 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48d983-d24c-4e56-93c1-b838515b037b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:39.376099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48d983-d24c-4e56-93c1-b838515b037b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:39.382340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 48/97] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:39 2022] GET /identity/v3/role_assignments?user.id=f0f33c461e67433593c4ca55f81f4e7e&scope.project.id=c13bf11626c248379284b9d7e67527bc&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:41.304600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ea61c-1164-4171-9548-2dd727eedddc None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:41.305619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ea61c-1164-4171-9548-2dd727eedddc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:41.306102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ea61c-1164-4171-9548-2dd727eedddc None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:41.307793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 50/98] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:41.313366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c05a4-83f1-4f09-855a-62759f5e1714 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:41.314313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c05a4-83f1-4f09-855a-62759f5e1714 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:41.314793 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c05a4-83f1-4f09-855a-62759f5e1714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:41.343360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0c1c05a4-83f1-4f09-855a-62759f5e1714 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:41.344990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0c1c05a4-83f1-4f09-855a-62759f5e1714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:41.354887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:41.391700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c3b4a-97d3-49df-b841-6207b28f6010 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:41.392327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c3b4a-97d3-49df-b841-6207b28f6010 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:41.392798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c3b4a-97d3-49df-b841-6207b28f6010 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:41.394925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 51/99] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:41.396184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7420188-c974-4bd5-b258-c238aa0710e9 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:41.396662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7420188-c974-4bd5-b258-c238aa0710e9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:41.397126 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7420188-c974-4bd5-b258-c238aa0710e9 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:41.398525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 52/100] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:41.399995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5885c-8ee8-44cb-a7ab-0a672350f34b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:41.400494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5885c-8ee8-44cb-a7ab-0a672350f34b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:41.400971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5885c-8ee8-44cb-a7ab-0a672350f34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:41.440293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f0d5885c-8ee8-44cb-a7ab-0a672350f34b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:41.444142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f0d5885c-8ee8-44cb-a7ab-0a672350f34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:41.458644 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:41.461871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 49/101] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:41.463403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd30e-4a9c-45a0-ad8c-26dfe93c3ea5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:41.463883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd30e-4a9c-45a0-ad8c-26dfe93c3ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:41.464405 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd30e-4a9c-45a0-ad8c-26dfe93c3ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:41.483159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fcacd30e-4a9c-45a0-ad8c-26dfe93c3ea5 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:41.484644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fcacd30e-4a9c-45a0-ad8c-26dfe93c3ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:41.556122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 50/102] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:41.557668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ddbee-032a-4229-bd64-b0e70999d3a1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:41.558196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ddbee-032a-4229-bd64-b0e70999d3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:41.558663 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ddbee-032a-4229-bd64-b0e70999d3a1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:41.560098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 51/103] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:41.565694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 53/104] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:41.565901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4bb76-cb90-4463-b6de-235d911df7bd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:41.566909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4bb76-cb90-4463-b6de-235d911df7bd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:41.567404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4bb76-cb90-4463-b6de-235d911df7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:41.586421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-aef4bb76-cb90-4463-b6de-235d911df7bd None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:41.587922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aef4bb76-cb90-4463-b6de-235d911df7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:41.659343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 52/105] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:41.788385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5885c-8ee8-44cb-a7ab-0a672350f34b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:41.870755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71571da-b496-4521-a520-79ec6b7bf4fb None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:41.871382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71571da-b496-4521-a520-79ec6b7bf4fb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:41.871874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71571da-b496-4521-a520-79ec6b7bf4fb None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:41.873750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 53/106] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:41.885931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa083eba-98aa-47bb-a1a3-12d905164fce None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:41.886521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa083eba-98aa-47bb-a1a3-12d905164fce None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:41.886990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa083eba-98aa-47bb-a1a3-12d905164fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:41.901882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:41.925251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5885c-8ee8-44cb-a7ab-0a672350f34b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:42.016917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fa083eba-98aa-47bb-a1a3-12d905164fce None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:42.018478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fa083eba-98aa-47bb-a1a3-12d905164fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:42.061973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a4475-437f-4c38-84c3-04648f9b0a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:42.062665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a4475-437f-4c38-84c3-04648f9b0a6f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.062943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a4475-437f-4c38-84c3-04648f9b0a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.063211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a4475-437f-4c38-84c3-04648f9b0a6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.064449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a4475-437f-4c38-84c3-04648f9b0a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:42.065877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a4475-437f-4c38-84c3-04648f9b0a6f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:42.068391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 54/107] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:42.069425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a4475-437f-4c38-84c3-04648f9b0a6f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:42.087288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:42.108737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 54/108] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:42.110685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa083eba-98aa-47bb-a1a3-12d905164fce None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:42.190007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a4475-437f-4c38-84c3-04648f9b0a6f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:42.225118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85e8b4-2382-4fb6-b10c-4309a8f2b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:42.226063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85e8b4-2382-4fb6-b10c-4309a8f2b299 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.226581 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85e8b4-2382-4fb6-b10c-4309a8f2b299 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.227048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85e8b4-2382-4fb6-b10c-4309a8f2b299 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.227504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa083eba-98aa-47bb-a1a3-12d905164fce None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:42.242090 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85e8b4-2382-4fb6-b10c-4309a8f2b299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-1b85e8b4-2382-4fb6-b10c-4309a8f2b299 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:42.249128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:42.249977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:42.251043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 55/109] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:23:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:42.256309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85e8b4-2382-4fb6-b10c-4309a8f2b299 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:42.266383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:42.271683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590899b8-6a31-4313-a8fd-0799f3d0be93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:42.272585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590899b8-6a31-4313-a8fd-0799f3d0be93 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.273166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590899b8-6a31-4313-a8fd-0799f3d0be93 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.273649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590899b8-6a31-4313-a8fd-0799f3d0be93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.286282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590899b8-6a31-4313-a8fd-0799f3d0be93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-590899b8-6a31-4313-a8fd-0799f3d0be93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:42.292650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:42.293479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:42.294325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:42.301684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 55/110] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:42.301684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590899b8-6a31-4313-a8fd-0799f3d0be93 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:42.359363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:42.387519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85e8b4-2382-4fb6-b10c-4309a8f2b299 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:42.420079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590899b8-6a31-4313-a8fd-0799f3d0be93 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:42.428867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bdd5d-4385-491c-b9d2-0da91e139c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:42.429741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bdd5d-4385-491c-b9d2-0da91e139c46 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.430369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bdd5d-4385-491c-b9d2-0da91e139c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.430837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bdd5d-4385-491c-b9d2-0da91e139c46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.442072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bdd5d-4385-491c-b9d2-0da91e139c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-fa3bdd5d-4385-491c-b9d2-0da91e139c46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:42.447854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:42.448864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:23:42.449777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:42.451997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 56/111] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:42.452838 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bdd5d-4385-491c-b9d2-0da91e139c46 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:42.457661 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d29df8-da7b-4631-8508-65e7dc872843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:42.458294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d29df8-da7b-4631-8508-65e7dc872843 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.458578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d29df8-da7b-4631-8508-65e7dc872843 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.458850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d29df8-da7b-4631-8508-65e7dc872843 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.459490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d29df8-da7b-4631-8508-65e7dc872843 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:42.460925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d29df8-da7b-4631-8508-65e7dc872843 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:42.553334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bdd5d-4385-491c-b9d2-0da91e139c46 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:42.562109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 56/112] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:23:42 2022] POST /identity/v3/projects => generated 302 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:23:42.562986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d29df8-da7b-4631-8508-65e7dc872843 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:42.599380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdf8eb-d535-416f-9034-356e16c64564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:42.600311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdf8eb-d535-416f-9034-356e16c64564 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.600730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdf8eb-d535-416f-9034-356e16c64564 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.601315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdf8eb-d535-416f-9034-356e16c64564 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.603151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdf8eb-d535-416f-9034-356e16c64564 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:42.603621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdf8eb-d535-416f-9034-356e16c64564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:42.605192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdf8eb-d535-416f-9034-356e16c64564 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:42.608491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:42.613905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:42.619981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 57/113] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:42.621314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdf8eb-d535-416f-9034-356e16c64564 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:42.676702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d29df8-da7b-4631-8508-65e7dc872843 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:42.717367 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d52a6-5b63-4c98-a298-306ed776ccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:42.717930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d52a6-5b63-4c98-a298-306ed776ccb1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.718239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d52a6-5b63-4c98-a298-306ed776ccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.718518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d52a6-5b63-4c98-a298-306ed776ccb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.719197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d52a6-5b63-4c98-a298-306ed776ccb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:42.719507 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d52a6-5b63-4c98-a298-306ed776ccb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:42.720901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d52a6-5b63-4c98-a298-306ed776ccb1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:42.731933 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 57/114] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:42.732785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d52a6-5b63-4c98-a298-306ed776ccb1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:42.745684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdf8eb-d535-416f-9034-356e16c64564 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:42.785795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3bd8b-1dde-4117-beca-86b7dd3c1760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:42.786488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3bd8b-1dde-4117-beca-86b7dd3c1760 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.786773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3bd8b-1dde-4117-beca-86b7dd3c1760 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.787049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3bd8b-1dde-4117-beca-86b7dd3c1760 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.797648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3bd8b-1dde-4117-beca-86b7dd3c1760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9ea3bd8b-1dde-4117-beca-86b7dd3c1760 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:42.802998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:42.804013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:42.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 58/115] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:23:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:42.805652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a7557-0d5c-4fe3-85ef-381db9d01e2e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.805652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a7557-0d5c-4fe3-85ef-381db9d01e2e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.805814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a7557-0d5c-4fe3-85ef-381db9d01e2e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.807253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 59/116] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:42.807952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a7557-0d5c-4fe3-85ef-381db9d01e2e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:42.852049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d52a6-5b63-4c98-a298-306ed776ccb1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:42.859826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:42.871578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:42.893323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218a3b0-1fc1-4ddf-8eda-22611436122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:42.894602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a3b0-1fc1-4ddf-8eda-22611436122a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.894901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a3b0-1fc1-4ddf-8eda-22611436122a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.895596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a3b0-1fc1-4ddf-8eda-22611436122a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.896650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218a3b0-1fc1-4ddf-8eda-22611436122a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:42.896650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218a3b0-1fc1-4ddf-8eda-22611436122a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:42.898034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218a3b0-1fc1-4ddf-8eda-22611436122a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:42.906721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 58/117] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:42.907610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218a3b0-1fc1-4ddf-8eda-22611436122a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:42.918500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a7557-0d5c-4fe3-85ef-381db9d01e2e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:42.954278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434a7c1-3d31-40e1-afe0-87eb50000cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:42.954923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434a7c1-3d31-40e1-afe0-87eb50000cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:42.955207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434a7c1-3d31-40e1-afe0-87eb50000cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:42.955485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434a7c1-3d31-40e1-afe0-87eb50000cd4 None admin] PATH_INFO: `/v3/users/4444aa1a4685411fad68fe8a8f2d8f11` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:42.957140 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434a7c1-3d31-40e1-afe0-87eb50000cd4 None admin] RBAC: Authorizing `identity:get_user(user_id=4444aa1a4685411fad68fe8a8f2d8f11)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:42.958646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434a7c1-3d31-40e1-afe0-87eb50000cd4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:42.962183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 60/118] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:42 2022] GET /identity/v3/users/4444aa1a4685411fad68fe8a8f2d8f11 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:42.962183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434a7c1-3d31-40e1-afe0-87eb50000cd4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:43.018783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218a3b0-1fc1-4ddf-8eda-22611436122a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:43.055082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3d3a6-6dba-47f8-9ed8-38b95edace4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:43.055716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3d3a6-6dba-47f8-9ed8-38b95edace4e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.055994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3d3a6-6dba-47f8-9ed8-38b95edace4e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.056267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3d3a6-6dba-47f8-9ed8-38b95edace4e None admin] PATH_INFO: `/v3/users/f0f33c461e67433593c4ca55f81f4e7e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.056778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434a7c1-3d31-40e1-afe0-87eb50000cd4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:43.057550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3d3a6-6dba-47f8-9ed8-38b95edace4e None admin] RBAC: Authorizing `identity:get_user(user_id=f0f33c461e67433593c4ca55f81f4e7e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:43.059406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3d3a6-6dba-47f8-9ed8-38b95edace4e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:43.060986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 59/119] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:42 2022] GET /identity/v3/users/f0f33c461e67433593c4ca55f81f4e7e => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:43.062088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c9a82-d84a-475a-ae62-1327a489f20a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.062390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c9a82-d84a-475a-ae62-1327a489f20a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.062658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c9a82-d84a-475a-ae62-1327a489f20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.082940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6b2c9a82-d84a-475a-ae62-1327a489f20a None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:43.084123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6b2c9a82-d84a-475a-ae62-1327a489f20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:43.086188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:43.094751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f09cb-b415-4fb5-80e5-41a5a785fabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:43.095411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f09cb-b415-4fb5-80e5-41a5a785fabb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.095706 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f09cb-b415-4fb5-80e5-41a5a785fabb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.095981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f09cb-b415-4fb5-80e5-41a5a785fabb None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.096645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f09cb-b415-4fb5-80e5-41a5a785fabb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:43.096952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f09cb-b415-4fb5-80e5-41a5a785fabb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:43.098411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f09cb-b415-4fb5-80e5-41a5a785fabb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:43.107304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 61/120] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:23:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:43.107953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f09cb-b415-4fb5-80e5-41a5a785fabb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:43.119101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:43.165125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 60/121] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:43.166433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c9a82-d84a-475a-ae62-1327a489f20a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:43.225952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f09cb-b415-4fb5-80e5-41a5a785fabb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:43.275210 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c20cd4-3e55-49b9-8ca8-3571aa5bf449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:43.275989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c20cd4-3e55-49b9-8ca8-3571aa5bf449 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.276477 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c20cd4-3e55-49b9-8ca8-3571aa5bf449 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.276819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c20cd4-3e55-49b9-8ca8-3571aa5bf449 None admin] PATH_INFO: `/v3/users/e7f18540e5e841e580c1915e9309de5c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.287939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c9a82-d84a-475a-ae62-1327a489f20a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:43.295560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c20cd4-3e55-49b9-8ca8-3571aa5bf449 None admin] RBAC: Authorizing `identity:get_user(user_id=e7f18540e5e841e580c1915e9309de5c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:43.297611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c20cd4-3e55-49b9-8ca8-3571aa5bf449 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:43.304316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 62/122] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:43 2022] GET /identity/v3/users/e7f18540e5e841e580c1915e9309de5c => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:43.304316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c20cd4-3e55-49b9-8ca8-3571aa5bf449 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:43.339186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e23e7-1dc8-4852-a0f9-2043c0d5a4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:43.339186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e23e7-1dc8-4852-a0f9-2043c0d5a4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.339186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e23e7-1dc8-4852-a0f9-2043c0d5a4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.339186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e23e7-1dc8-4852-a0f9-2043c0d5a4d9 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.339567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e23e7-1dc8-4852-a0f9-2043c0d5a4d9 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:43.339567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e23e7-1dc8-4852-a0f9-2043c0d5a4d9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:43.339567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 61/123] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:43 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:43.340232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e23e7-1dc8-4852-a0f9-2043c0d5a4d9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:43.360560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:43.385682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:43.416372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c20cd4-3e55-49b9-8ca8-3571aa5bf449 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:43.454992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5efa1d-63a6-4001-ae43-444a3ee38007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:43.456039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5efa1d-63a6-4001-ae43-444a3ee38007 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.456465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5efa1d-63a6-4001-ae43-444a3ee38007 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.456884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5efa1d-63a6-4001-ae43-444a3ee38007 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.458270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5efa1d-63a6-4001-ae43-444a3ee38007 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:43.460338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5efa1d-63a6-4001-ae43-444a3ee38007 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:43.461219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e23e7-1dc8-4852-a0f9-2043c0d5a4d9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:43.462100 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 63/124] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:43 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:43.466259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5efa1d-63a6-4001-ae43-444a3ee38007 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:43.500774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b2342-6686-4756-b96c-53593a349b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:43.501401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b2342-6686-4756-b96c-53593a349b8a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.501692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b2342-6686-4756-b96c-53593a349b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.501946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b2342-6686-4756-b96c-53593a349b8a None admin] PATH_INFO: `/v3/users/f0f33c461e67433593c4ca55f81f4e7e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.503283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b2342-6686-4756-b96c-53593a349b8a None admin] RBAC: Authorizing `identity:get_user(user_id=f0f33c461e67433593c4ca55f81f4e7e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:43.505196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b2342-6686-4756-b96c-53593a349b8a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:43.511337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 62/125] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:43 2022] GET /identity/v3/users/f0f33c461e67433593c4ca55f81f4e7e => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:43.511337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b2342-6686-4756-b96c-53593a349b8a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:43.577890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5efa1d-63a6-4001-ae43-444a3ee38007 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:43.590953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:43.611975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7061fe-bb4b-4ee6-b171-d0d7e0f9c712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:43.612240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b2342-6686-4756-b96c-53593a349b8a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:43.612591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7061fe-bb4b-4ee6-b171-d0d7e0f9c712 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.613393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7061fe-bb4b-4ee6-b171-d0d7e0f9c712 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.613670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7061fe-bb4b-4ee6-b171-d0d7e0f9c712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.614397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:43.648491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5ec02-5025-4d4a-9cda-ceb42711a324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:43.649403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5ec02-5025-4d4a-9cda-ceb42711a324 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.650410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5ec02-5025-4d4a-9cda-ceb42711a324 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.650708 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5ec02-5025-4d4a-9cda-ceb42711a324 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/users/4444aa1a4685411fad68fe8a8f2d8f11/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.652632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5ec02-5025-4d4a-9cda-ceb42711a324 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user_id=4444aa1a4685411fad68fe8a8f2d8f11, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:43.654065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5ec02-5025-4d4a-9cda-ceb42711a324 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:43.683013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7061fe-bb4b-4ee6-b171-d0d7e0f9c712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:43.684464 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7061fe-bb4b-4ee6-b171-d0d7e0f9c712 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:43.731672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 63/126] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:23:43 2022] PUT /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/users/4444aa1a4685411fad68fe8a8f2d8f11/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:23:43.732765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5ec02-5025-4d4a-9cda-ceb42711a324 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:43.813373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 64/127] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:23:43.813373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7061fe-bb4b-4ee6-b171-d0d7e0f9c712 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:43.832493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:43.902168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5ec02-5025-4d4a-9cda-ceb42711a324 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:43.948945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7061fe-bb4b-4ee6-b171-d0d7e0f9c712 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:43.951761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7513de-6783-49a5-81c6-471031349333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:43.952630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7513de-6783-49a5-81c6-471031349333 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.953131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7513de-6783-49a5-81c6-471031349333 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.953620 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7513de-6783-49a5-81c6-471031349333 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.965143 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7513de-6783-49a5-81c6-471031349333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-be7513de-6783-49a5-81c6-471031349333 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:43.970596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:43.971453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:43.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 64/128] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:43.974058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7513de-6783-49a5-81c6-471031349333 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:43.989915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac35787-fa0f-46aa-aaf1-cc498099ea95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:43.990746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35787-fa0f-46aa-aaf1-cc498099ea95 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:43.991119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35787-fa0f-46aa-aaf1-cc498099ea95 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:43.991491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35787-fa0f-46aa-aaf1-cc498099ea95 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/users/f0f33c461e67433593c4ca55f81f4e7e/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:43.993501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac35787-fa0f-46aa-aaf1-cc498099ea95 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user_id=f0f33c461e67433593c4ca55f81f4e7e, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:43.995088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac35787-fa0f-46aa-aaf1-cc498099ea95 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:44.004528 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:44.081921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7513de-6783-49a5-81c6-471031349333 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:44.093735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 65/129] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:23:43 2022] PUT /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/users/f0f33c461e67433593c4ca55f81f4e7e/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:23:44.095025 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac35787-fa0f-46aa-aaf1-cc498099ea95 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:44.140373 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926da16-357b-43ac-81fd-b733d530b955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:44.141763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926da16-357b-43ac-81fd-b733d530b955 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:44.142060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926da16-357b-43ac-81fd-b733d530b955 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:44.142527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926da16-357b-43ac-81fd-b733d530b955 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:44.144516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926da16-357b-43ac-81fd-b733d530b955 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:44.146125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926da16-357b-43ac-81fd-b733d530b955 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:44.149128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 65/130] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:43 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:44.149997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926da16-357b-43ac-81fd-b733d530b955 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:44.257641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:44.308545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac35787-fa0f-46aa-aaf1-cc498099ea95 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:44.313960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:44.324860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926da16-357b-43ac-81fd-b733d530b955 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:44.367657 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f26f5e-c367-419d-b760-6b58c2bed200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:44.367830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d09e-c478-4df4-b609-403c8bc47c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:44.368429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d09e-c478-4df4-b609-403c8bc47c17 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:44.368620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d09e-c478-4df4-b609-403c8bc47c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:44.368902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d09e-c478-4df4-b609-403c8bc47c17 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:44.369627 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f26f5e-c367-419d-b760-6b58c2bed200 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:44.370183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f26f5e-c367-419d-b760-6b58c2bed200 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:44.370616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7d09e-c478-4df4-b609-403c8bc47c17 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:44.370856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f26f5e-c367-419d-b760-6b58c2bed200 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:44.371732 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f26f5e-c367-419d-b760-6b58c2bed200 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:44.372057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7d09e-c478-4df4-b609-403c8bc47c17 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:44.372424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f26f5e-c367-419d-b760-6b58c2bed200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:44.373976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f26f5e-c367-419d-b760-6b58c2bed200 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:44.374078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 66/131] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:44 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:44.374755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d09e-c478-4df4-b609-403c8bc47c17 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:44.388084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 66/132] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:44.389531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f26f5e-c367-419d-b760-6b58c2bed200 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:44.500497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d09e-c478-4df4-b609-403c8bc47c17 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:44.544190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f26f5e-c367-419d-b760-6b58c2bed200 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:44.573988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b0410-8014-421f-af05-215cb9e80737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:44.575864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b0410-8014-421f-af05-215cb9e80737 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:44.577622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b0410-8014-421f-af05-215cb9e80737 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:44.578100 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b0410-8014-421f-af05-215cb9e80737 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/users/f0f33c461e67433593c4ca55f81f4e7e/roles/cae161e9c1b64656924448dd784ed4d1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:44.583008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b0410-8014-421f-af05-215cb9e80737 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user_id=f0f33c461e67433593c4ca55f81f4e7e, role_id=cae161e9c1b64656924448dd784ed4d1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:44.586900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b0410-8014-421f-af05-215cb9e80737 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:44.605706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9910bd7-9f18-443a-8863-d702d94dc376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:44.606692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9910bd7-9f18-443a-8863-d702d94dc376 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:44.607171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9910bd7-9f18-443a-8863-d702d94dc376 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:44.607661 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9910bd7-9f18-443a-8863-d702d94dc376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:44.608583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9910bd7-9f18-443a-8863-d702d94dc376 None admin] RBAC: Adding query filter params (role.id=840afb6280c341c6beefc211b040c463, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user.id=e7f18540e5e841e580c1915e9309de5c) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:44.609089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9910bd7-9f18-443a-8863-d702d94dc376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:44.610851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9910bd7-9f18-443a-8863-d702d94dc376 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:44.616952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 67/133] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:44 2022] GET /identity/v3/role_assignments?user.id=e7f18540e5e841e580c1915e9309de5c&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=840afb6280c341c6beefc211b040c463 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:44.618131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9910bd7-9f18-443a-8863-d702d94dc376 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:44.640011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:44.664181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 67/134] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:23:44 2022] PUT /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/users/f0f33c461e67433593c4ca55f81f4e7e/roles/cae161e9c1b64656924448dd784ed4d1 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:23:44.746178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9910bd7-9f18-443a-8863-d702d94dc376 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:44.788174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e620bc4-b238-45d5-bc33-bdab2505ca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:44.788838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e620bc4-b238-45d5-bc33-bdab2505ca62 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:44.789115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e620bc4-b238-45d5-bc33-bdab2505ca62 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:44.789390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e620bc4-b238-45d5-bc33-bdab2505ca62 None admin] PATH_INFO: `/v3/users/f0f33c461e67433593c4ca55f81f4e7e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:44.790571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e620bc4-b238-45d5-bc33-bdab2505ca62 None admin] RBAC: Authorizing `identity:get_user(user_id=f0f33c461e67433593c4ca55f81f4e7e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:44.792022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e620bc4-b238-45d5-bc33-bdab2505ca62 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:44.794129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 68/135] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:44 2022] GET /identity/v3/users/f0f33c461e67433593c4ca55f81f4e7e => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:44.799899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b0410-8014-421f-af05-215cb9e80737 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:44.808626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:44.933503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28c9be-f8e0-4a1d-a6be-7924a0b04c84 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:44.934166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28c9be-f8e0-4a1d-a6be-7924a0b04c84 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:44.934672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28c9be-f8e0-4a1d-a6be-7924a0b04c84 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:44.936235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 69/136] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:44.942879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703194d9-bcb7-4add-80ec-7ea85539675a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:44.943737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703194d9-bcb7-4add-80ec-7ea85539675a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:44.944205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703194d9-bcb7-4add-80ec-7ea85539675a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:44.955838 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b0410-8014-421f-af05-215cb9e80737 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:44.966483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-703194d9-bcb7-4add-80ec-7ea85539675a None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:44.968548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-703194d9-bcb7-4add-80ec-7ea85539675a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:45.022928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96f7d-5b57-497b-9ea0-b3a9ef74ae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:45.023981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96f7d-5b57-497b-9ea0-b3a9ef74ae00 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:45.024979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96f7d-5b57-497b-9ea0-b3a9ef74ae00 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:45.025456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd96f7d-5b57-497b-9ea0-b3a9ef74ae00 None admin] PATH_INFO: `/v3/projects/c13bf11626c248379284b9d7e67527bc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:45.027585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd96f7d-5b57-497b-9ea0-b3a9ef74ae00 None admin] RBAC: Authorizing `identity:get_project(project_id=c13bf11626c248379284b9d7e67527bc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:45.029802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd96f7d-5b57-497b-9ea0-b3a9ef74ae00 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:45.032935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 68/137] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:44 2022] GET /identity/v3/projects/c13bf11626c248379284b9d7e67527bc => generated 312 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:45.043127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96f7d-5b57-497b-9ea0-b3a9ef74ae00 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:45.095147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 70/138] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:23:45.137161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:45.173718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd96f7d-5b57-497b-9ea0-b3a9ef74ae00 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:45.213269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adef63-a6c9-41ec-bca8-9f10d91d26b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:45.213944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adef63-a6c9-41ec-bca8-9f10d91d26b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:45.214261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adef63-a6c9-41ec-bca8-9f10d91d26b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:45.214537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adef63-a6c9-41ec-bca8-9f10d91d26b4 None admin] PATH_INFO: `/v3/projects/c13bf11626c248379284b9d7e67527bc/users/f0f33c461e67433593c4ca55f81f4e7e/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:45.216443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adef63-a6c9-41ec-bca8-9f10d91d26b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c13bf11626c248379284b9d7e67527bc, user_id=f0f33c461e67433593c4ca55f81f4e7e, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:45.217882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adef63-a6c9-41ec-bca8-9f10d91d26b4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:45.310039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 69/139] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:23:45 2022] PUT /identity/v3/projects/c13bf11626c248379284b9d7e67527bc/users/f0f33c461e67433593c4ca55f81f4e7e/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:23:45.469149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703194d9-bcb7-4add-80ec-7ea85539675a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:45.485517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:45.586598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703194d9-bcb7-4add-80ec-7ea85539675a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:45.622950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d93d69-5dcc-43dc-ab16-3e408b16cacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:45.623596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d93d69-5dcc-43dc-ab16-3e408b16cacb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:45.623881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d93d69-5dcc-43dc-ab16-3e408b16cacb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:45.624160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d93d69-5dcc-43dc-ab16-3e408b16cacb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:45.625292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d93d69-5dcc-43dc-ab16-3e408b16cacb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:45.626781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d93d69-5dcc-43dc-ab16-3e408b16cacb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:45.632501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 71/140] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:45.634575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adef63-a6c9-41ec-bca8-9f10d91d26b4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:45.741567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adef63-a6c9-41ec-bca8-9f10d91d26b4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:45.771380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:45.777784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a687177-511d-43ec-a16a-6b70f363f887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:45.778527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a687177-511d-43ec-a16a-6b70f363f887 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:45.778872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a687177-511d-43ec-a16a-6b70f363f887 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:45.779201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a687177-511d-43ec-a16a-6b70f363f887 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:45.779881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a687177-511d-43ec-a16a-6b70f363f887 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:45.781389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a687177-511d-43ec-a16a-6b70f363f887 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:45.904662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 70/141] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:23:45 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:23:46.838171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19656e-b2eb-4100-a9c8-40f54f3dcfa3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:46.838687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19656e-b2eb-4100-a9c8-40f54f3dcfa3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:46.838946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19656e-b2eb-4100-a9c8-40f54f3dcfa3 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:46.840562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 72/142] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:46.845468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c465671-73f8-47df-91d8-a7c52ac36ea3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:46.845950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c465671-73f8-47df-91d8-a7c52ac36ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:46.846024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c465671-73f8-47df-91d8-a7c52ac36ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:46.873926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2c465671-73f8-47df-91d8-a7c52ac36ea3 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:46.875163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2c465671-73f8-47df-91d8-a7c52ac36ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:46.965120 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 71/143] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:23:47.029962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396316b-52e5-41f2-b6cf-96e703e1f1ac None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:47.030273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396316b-52e5-41f2-b6cf-96e703e1f1ac None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:47.030559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396316b-52e5-41f2-b6cf-96e703e1f1ac None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:47.031977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 73/144] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:47.037439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aa047-a679-4945-9eeb-7016a29851e4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:47.037739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aa047-a679-4945-9eeb-7016a29851e4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:47.038036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aa047-a679-4945-9eeb-7016a29851e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:47.060921 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b66aa047-a679-4945-9eeb-7016a29851e4 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:47.062275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b66aa047-a679-4945-9eeb-7016a29851e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:47.140948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 72/145] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:47.338035 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396316b-52e5-41f2-b6cf-96e703e1f1ac None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:47.432770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:47.452738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396316b-52e5-41f2-b6cf-96e703e1f1ac None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:47.490739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121737d-5f46-4cc5-9c88-348b093dbf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:47.491383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121737d-5f46-4cc5-9c88-348b093dbf3d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:47.491667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121737d-5f46-4cc5-9c88-348b093dbf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:47.491943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121737d-5f46-4cc5-9c88-348b093dbf3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:47.502536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121737d-5f46-4cc5-9c88-348b093dbf3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2121737d-5f46-4cc5-9c88-348b093dbf3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:47.508259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:47.509086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:47.509825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:47.511838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 74/146] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:47.515408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aa047-a679-4945-9eeb-7016a29851e4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:47.598029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:47.658321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aa047-a679-4945-9eeb-7016a29851e4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:47.701351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9f5d6-e220-4286-a36d-c22adb6a9ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:47.701885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9f5d6-e220-4286-a36d-c22adb6a9ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:47.702181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9f5d6-e220-4286-a36d-c22adb6a9ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:47.702459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9f5d6-e220-4286-a36d-c22adb6a9ec0 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:47.703129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9f5d6-e220-4286-a36d-c22adb6a9ec0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:47.703430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9f5d6-e220-4286-a36d-c22adb6a9ec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:47.704824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9f5d6-e220-4286-a36d-c22adb6a9ec0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:47.718061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 73/147] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:47.723907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121737d-5f46-4cc5-9c88-348b093dbf3d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:47.773758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9f5d6-e220-4286-a36d-c22adb6a9ec0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:47.897228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121737d-5f46-4cc5-9c88-348b093dbf3d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:47.963156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9f5d6-e220-4286-a36d-c22adb6a9ec0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:47.973004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:47.980692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099142e7-0766-4f89-a6c6-173ccebd6cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:47.981597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099142e7-0766-4f89-a6c6-173ccebd6cdf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:47.982067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099142e7-0766-4f89-a6c6-173ccebd6cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:47.982588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099142e7-0766-4f89-a6c6-173ccebd6cdf None admin] PATH_INFO: `/v3/users/f0f33c461e67433593c4ca55f81f4e7e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:47.984982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099142e7-0766-4f89-a6c6-173ccebd6cdf None admin] RBAC: Authorizing `identity:get_user(user_id=f0f33c461e67433593c4ca55f81f4e7e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:47.986726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099142e7-0766-4f89-a6c6-173ccebd6cdf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:47.988588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 75/148] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:47 2022] GET /identity/v3/users/f0f33c461e67433593c4ca55f81f4e7e => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:47.989920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ad64a-8455-4648-9297-dbcb00320422 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:47.994576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ad64a-8455-4648-9297-dbcb00320422 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:47.995076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ad64a-8455-4648-9297-dbcb00320422 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:47.996583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 76/149] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:47.997525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ad64a-8455-4648-9297-dbcb00320422 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:48.006852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557ef8e-8148-4da0-b8e8-3f578a9f0659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:48.007512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557ef8e-8148-4da0-b8e8-3f578a9f0659 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:48.007791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557ef8e-8148-4da0-b8e8-3f578a9f0659 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:48.008149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557ef8e-8148-4da0-b8e8-3f578a9f0659 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:48.018500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:48.032125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557ef8e-8148-4da0-b8e8-3f578a9f0659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8557ef8e-8148-4da0-b8e8-3f578a9f0659 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:48.039819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:48.040685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 74/150] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:23:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:48.041461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df10d7-af71-4f19-91ce-2ab209a66257 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:48.042132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df10d7-af71-4f19-91ce-2ab209a66257 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:48.042132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df10d7-af71-4f19-91ce-2ab209a66257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:48.077162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-88df10d7-af71-4f19-91ce-2ab209a66257 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:48.078186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-88df10d7-af71-4f19-91ce-2ab209a66257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:48.151934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ad64a-8455-4648-9297-dbcb00320422 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:48.200245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 75/151] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:48.201321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df10d7-af71-4f19-91ce-2ab209a66257 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:48.210282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafac5ec-ab48-4cd6-b1ea-cbba9943e770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:48.211203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafac5ec-ab48-4cd6-b1ea-cbba9943e770 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:48.211671 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafac5ec-ab48-4cd6-b1ea-cbba9943e770 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:48.212148 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafac5ec-ab48-4cd6-b1ea-cbba9943e770 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:48.294758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafac5ec-ab48-4cd6-b1ea-cbba9943e770 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:48.296552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafac5ec-ab48-4cd6-b1ea-cbba9943e770 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:48.311202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:48.387104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:48.392731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df10d7-af71-4f19-91ce-2ab209a66257 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:48.403632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 77/152] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:23:48.405219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76da985-f48e-4d69-aacd-02831d83fd59 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:48.405763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76da985-f48e-4d69-aacd-02831d83fd59 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:48.406284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76da985-f48e-4d69-aacd-02831d83fd59 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:48.407802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 78/153] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:48.415764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463ea83-dbff-48ac-92c9-3a6b02734dd4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:48.416326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463ea83-dbff-48ac-92c9-3a6b02734dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:48.416791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463ea83-dbff-48ac-92c9-3a6b02734dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:48.444412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f463ea83-dbff-48ac-92c9-3a6b02734dd4 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:48.446031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f463ea83-dbff-48ac-92c9-3a6b02734dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:48.471939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a42f99-4e7b-4560-925c-30a395b5260e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:48.472611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a42f99-4e7b-4560-925c-30a395b5260e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:48.472893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a42f99-4e7b-4560-925c-30a395b5260e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:48.473167 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a42f99-4e7b-4560-925c-30a395b5260e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:48.473851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a42f99-4e7b-4560-925c-30a395b5260e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:48.474570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a42f99-4e7b-4560-925c-30a395b5260e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:48.475678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a42f99-4e7b-4560-925c-30a395b5260e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:48.494034 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 76/154] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:48.494915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a42f99-4e7b-4560-925c-30a395b5260e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:48.532668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 79/155] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:48.533955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463ea83-dbff-48ac-92c9-3a6b02734dd4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:48.580011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:48.646401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463ea83-dbff-48ac-92c9-3a6b02734dd4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:48.681521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a42f99-4e7b-4560-925c-30a395b5260e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:48.687493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfd1c-ae6a-4df9-be2b-91f59ac77b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:48.688260 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfd1c-ae6a-4df9-be2b-91f59ac77b8a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:48.688646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfd1c-ae6a-4df9-be2b-91f59ac77b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:48.688997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfd1c-ae6a-4df9-be2b-91f59ac77b8a None admin] PATH_INFO: `/v3/users/4444aa1a4685411fad68fe8a8f2d8f11` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:48.690273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cfd1c-ae6a-4df9-be2b-91f59ac77b8a None admin] RBAC: Authorizing `identity:get_user(user_id=4444aa1a4685411fad68fe8a8f2d8f11)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:48.691987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cfd1c-ae6a-4df9-be2b-91f59ac77b8a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:48.694509 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 80/156] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:48 2022] GET /identity/v3/users/4444aa1a4685411fad68fe8a8f2d8f11 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:48.701222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfd1c-ae6a-4df9-be2b-91f59ac77b8a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:48.734282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd125bff-ff4f-405c-8b97-886f6d21755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:48.735253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd125bff-ff4f-405c-8b97-886f6d21755d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:48.735683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd125bff-ff4f-405c-8b97-886f6d21755d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:48.736160 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd125bff-ff4f-405c-8b97-886f6d21755d None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:48.737567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd125bff-ff4f-405c-8b97-886f6d21755d None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:48.739364 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd125bff-ff4f-405c-8b97-886f6d21755d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:48.741363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 77/157] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:48 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:48.747169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd125bff-ff4f-405c-8b97-886f6d21755d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:48.794478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:48.815018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfd1c-ae6a-4df9-be2b-91f59ac77b8a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:48.826568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:48.854624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdf8a8-4e59-400a-8d5e-773b68e438e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:48.855441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdf8a8-4e59-400a-8d5e-773b68e438e8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:48.856082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdf8a8-4e59-400a-8d5e-773b68e438e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:48.856356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdf8a8-4e59-400a-8d5e-773b68e438e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:48.920965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd125bff-ff4f-405c-8b97-886f6d21755d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:48.932508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdf8a8-4e59-400a-8d5e-773b68e438e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:48.934090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdf8a8-4e59-400a-8d5e-773b68e438e8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:48.960653 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5ec5f-2665-4e62-bb72-f1ffd51509c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:48.961976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5ec5f-2665-4e62-bb72-f1ffd51509c8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:48.962928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5ec5f-2665-4e62-bb72-f1ffd51509c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:48.963652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5ec5f-2665-4e62-bb72-f1ffd51509c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:48.965252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5ec5f-2665-4e62-bb72-f1ffd51509c8 None admin] RBAC: Adding query filter params (role.id=1e04fdc0cc334fd7a733a1490a194e37, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user.id=f0f33c461e67433593c4ca55f81f4e7e) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:48.966173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5ec5f-2665-4e62-bb72-f1ffd51509c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:48.968362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5ec5f-2665-4e62-bb72-f1ffd51509c8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:48.980425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 78/158] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:48 2022] GET /identity/v3/role_assignments?user.id=f0f33c461e67433593c4ca55f81f4e7e&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:48.981253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5ec5f-2665-4e62-bb72-f1ffd51509c8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:49.065346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 81/159] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:23:49.075372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdf8a8-4e59-400a-8d5e-773b68e438e8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:49.094707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:49.231987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:49.232346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdf8a8-4e59-400a-8d5e-773b68e438e8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:49.246611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5ec5f-2665-4e62-bb72-f1ffd51509c8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:49.285535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6f4b7-c68e-42f3-ae14-29af8120aa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:49.288809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6f4b7-c68e-42f3-ae14-29af8120aa04 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:49.289354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6f4b7-c68e-42f3-ae14-29af8120aa04 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:49.289829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6f4b7-c68e-42f3-ae14-29af8120aa04 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:49.291309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6f4b7-c68e-42f3-ae14-29af8120aa04 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:49.292956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6f4b7-c68e-42f3-ae14-29af8120aa04 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:49.294899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 82/160] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:49 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:49.299711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6f4b7-c68e-42f3-ae14-29af8120aa04 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:49.310854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c47621-1236-466e-a644-28ba70d3c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:49.311522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c47621-1236-466e-a644-28ba70d3c9cf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:49.311801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c47621-1236-466e-a644-28ba70d3c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:49.312074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c47621-1236-466e-a644-28ba70d3c9cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:49.334194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c47621-1236-466e-a644-28ba70d3c9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-57c47621-1236-466e-a644-28ba70d3c9cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:49.336664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:49.337637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:49.342503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 79/161] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:23:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:49.343149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c47621-1236-466e-a644-28ba70d3c9cf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:49.448709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:49.514500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c47621-1236-466e-a644-28ba70d3c9cf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:49.538750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6f4b7-c68e-42f3-ae14-29af8120aa04 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:49.584441 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba55da0-64d6-444c-8ad8-e2e923240ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:49.586938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba55da0-64d6-444c-8ad8-e2e923240ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:49.587425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba55da0-64d6-444c-8ad8-e2e923240ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:49.587923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba55da0-64d6-444c-8ad8-e2e923240ae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:49.589297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba55da0-64d6-444c-8ad8-e2e923240ae0 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user.id=4444aa1a4685411fad68fe8a8f2d8f11) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:49.594525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba55da0-64d6-444c-8ad8-e2e923240ae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:49.598209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba55da0-64d6-444c-8ad8-e2e923240ae0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:49.610827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 80/162] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:49 2022] GET /identity/v3/role_assignments?user.id=4444aa1a4685411fad68fe8a8f2d8f11&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:49.612119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba55da0-64d6-444c-8ad8-e2e923240ae0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:49.643463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:49.646929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef2156-5190-4a7d-84e5-7264c48bf3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:49.647602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef2156-5190-4a7d-84e5-7264c48bf3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:49.647882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef2156-5190-4a7d-84e5-7264c48bf3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:49.648153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef2156-5190-4a7d-84e5-7264c48bf3d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:49.676797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef2156-5190-4a7d-84e5-7264c48bf3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-69ef2156-5190-4a7d-84e5-7264c48bf3d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:49.681721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:49.684742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 83/163] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:49.685689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874558e2-f28c-4426-967b-c903bdce6f05 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:49.686895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874558e2-f28c-4426-967b-c903bdce6f05 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:49.686895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874558e2-f28c-4426-967b-c903bdce6f05 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:49.686895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 84/164] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:49.695189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874558e2-f28c-4426-967b-c903bdce6f05 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:49.795817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba55da0-64d6-444c-8ad8-e2e923240ae0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:49.855243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037d555-668c-4a87-90bb-60b6a578213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:49.855865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037d555-668c-4a87-90bb-60b6a578213e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:49.856139 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037d555-668c-4a87-90bb-60b6a578213e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:49.856410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037d555-668c-4a87-90bb-60b6a578213e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:49.857076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037d555-668c-4a87-90bb-60b6a578213e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:49.857376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037d555-668c-4a87-90bb-60b6a578213e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:49.860283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037d555-668c-4a87-90bb-60b6a578213e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:49.869643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 81/165] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:23:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:49.871303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3481e-5b47-4667-9ff3-4112e681755e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:49.871597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3481e-5b47-4667-9ff3-4112e681755e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:49.871975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3481e-5b47-4667-9ff3-4112e681755e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:49.895845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-71f3481e-5b47-4667-9ff3-4112e681755e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:49.897167 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-71f3481e-5b47-4667-9ff3-4112e681755e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:49.970650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874558e2-f28c-4426-967b-c903bdce6f05 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:49.977783 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 82/166] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:49.978746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3481e-5b47-4667-9ff3-4112e681755e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:49.991160 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:50.060760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf85ade-557f-4741-977e-7d53a05fafdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:50.061284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf85ade-557f-4741-977e-7d53a05fafdc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:50.061562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf85ade-557f-4741-977e-7d53a05fafdc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:50.061834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf85ade-557f-4741-977e-7d53a05fafdc None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:50.062526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf85ade-557f-4741-977e-7d53a05fafdc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:50.062833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf85ade-557f-4741-977e-7d53a05fafdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:50.064202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf85ade-557f-4741-977e-7d53a05fafdc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:50.071540 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:50.081350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 85/167] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:50.088283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf85ade-557f-4741-977e-7d53a05fafdc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:50.101187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3481e-5b47-4667-9ff3-4112e681755e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:50.148546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695f034-6f8c-4d65-abd4-5cc074fdde4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:50.149537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695f034-6f8c-4d65-abd4-5cc074fdde4e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:50.149988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695f034-6f8c-4d65-abd4-5cc074fdde4e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:50.150538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695f034-6f8c-4d65-abd4-5cc074fdde4e None admin] PATH_INFO: `/v3/users/f0f33c461e67433593c4ca55f81f4e7e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:50.152242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695f034-6f8c-4d65-abd4-5cc074fdde4e None admin] RBAC: Authorizing `identity:get_user(user_id=f0f33c461e67433593c4ca55f81f4e7e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:50.154235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695f034-6f8c-4d65-abd4-5cc074fdde4e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:50.155998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 83/168] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:49 2022] GET /identity/v3/users/f0f33c461e67433593c4ca55f81f4e7e => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:50.162859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695f034-6f8c-4d65-abd4-5cc074fdde4e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:50.265459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:50.297884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695f034-6f8c-4d65-abd4-5cc074fdde4e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:50.316220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf85ade-557f-4741-977e-7d53a05fafdc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:50.354886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8eccba-755d-4004-80d2-807c5bdc1f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:50.355626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8eccba-755d-4004-80d2-807c5bdc1f61 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:50.356013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8eccba-755d-4004-80d2-807c5bdc1f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:50.356206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8eccba-755d-4004-80d2-807c5bdc1f61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:50.357846 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e82bc-8a72-4d03-a71d-3711a7739837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:50.358711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e82bc-8a72-4d03-a71d-3711a7739837 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:50.359190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e82bc-8a72-4d03-a71d-3711a7739837 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:50.359655 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e82bc-8a72-4d03-a71d-3711a7739837 None admin] PATH_INFO: `/v3/users/e7f18540e5e841e580c1915e9309de5c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:50.360971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e82bc-8a72-4d03-a71d-3711a7739837 None admin] RBAC: Authorizing `identity:get_user(user_id=e7f18540e5e841e580c1915e9309de5c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:50.362879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e82bc-8a72-4d03-a71d-3711a7739837 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:50.365200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 86/169] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:50 2022] GET /identity/v3/users/e7f18540e5e841e580c1915e9309de5c => generated 289 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:50.371081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e82bc-8a72-4d03-a71d-3711a7739837 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:50.489902 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:50.502533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8eccba-755d-4004-80d2-807c5bdc1f61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:50.507740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8eccba-755d-4004-80d2-807c5bdc1f61 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:50.560677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e82bc-8a72-4d03-a71d-3711a7739837 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:50.625305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb1453-78b9-4c89-9557-65736b0e067d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:50.625952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb1453-78b9-4c89-9557-65736b0e067d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:50.626256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb1453-78b9-4c89-9557-65736b0e067d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:50.626531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb1453-78b9-4c89-9557-65736b0e067d None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:50.627698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb1453-78b9-4c89-9557-65736b0e067d None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:50.629679 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb1453-78b9-4c89-9557-65736b0e067d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:50.631363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 87/170] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:50 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:50.632209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb1453-78b9-4c89-9557-65736b0e067d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:50.645098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:50.677750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 84/171] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:23:50.679501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0138a74-6a8c-4533-b560-aa4532a0d8bb None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:50.680141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0138a74-6a8c-4533-b560-aa4532a0d8bb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:50.680843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0138a74-6a8c-4533-b560-aa4532a0d8bb None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:50.682232 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 85/172] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:50.686839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0138a74-6a8c-4533-b560-aa4532a0d8bb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:50.751908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb1453-78b9-4c89-9557-65736b0e067d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:50.796875 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dcd49-de65-407a-a8b5-0ae4b9d60064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:50.797984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dcd49-de65-407a-a8b5-0ae4b9d60064 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:50.798300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dcd49-de65-407a-a8b5-0ae4b9d60064 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:50.798626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dcd49-de65-407a-a8b5-0ae4b9d60064 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:50.801112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:50.811980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dcd49-de65-407a-a8b5-0ae4b9d60064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ff4dcd49-de65-407a-a8b5-0ae4b9d60064 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:50.819195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:50.820128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:50.820946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:50.822210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 88/173] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:50.823194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dcd49-de65-407a-a8b5-0ae4b9d60064 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:50.829496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0138a74-6a8c-4533-b560-aa4532a0d8bb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:50.866789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef70b1-1043-4c08-8263-21a773ca1c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:50.867871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef70b1-1043-4c08-8263-21a773ca1c1f None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:50.867871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef70b1-1043-4c08-8263-21a773ca1c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:50.868085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef70b1-1043-4c08-8263-21a773ca1c1f None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/users/e7f18540e5e841e580c1915e9309de5c/roles/840afb6280c341c6beefc211b040c463` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:50.870096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef70b1-1043-4c08-8263-21a773ca1c1f None admin] RBAC: Authorizing `identity:create_grant(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user_id=e7f18540e5e841e580c1915e9309de5c, role_id=840afb6280c341c6beefc211b040c463)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:50.871678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef70b1-1043-4c08-8263-21a773ca1c1f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:50.942987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dcd49-de65-407a-a8b5-0ae4b9d60064 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:50.976890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 86/174] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:23:50 2022] PUT /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/users/e7f18540e5e841e580c1915e9309de5c/roles/840afb6280c341c6beefc211b040c463 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:23:50.979324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58eee7a-15b7-4ba4-8b47-d2116e0c8051 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:50.980415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58eee7a-15b7-4ba4-8b47-d2116e0c8051 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:50.980941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58eee7a-15b7-4ba4-8b47-d2116e0c8051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:50.981994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f4786-7949-41d0-9acf-fd9fea0252f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:50.982736 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f4786-7949-41d0-9acf-fd9fea0252f4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:50.983024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f4786-7949-41d0-9acf-fd9fea0252f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:50.983356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f4786-7949-41d0-9acf-fd9fea0252f4 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:50.984693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f4786-7949-41d0-9acf-fd9fea0252f4 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:50.986219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f4786-7949-41d0-9acf-fd9fea0252f4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:50.988792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 89/175] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:50 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:50.989713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f4786-7949-41d0-9acf-fd9fea0252f4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:51.007374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c58eee7a-15b7-4ba4-8b47-d2116e0c8051 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:51.008998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c58eee7a-15b7-4ba4-8b47-d2116e0c8051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:51.046311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:51.094907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 87/176] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:51.096162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58eee7a-15b7-4ba4-8b47-d2116e0c8051 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:51.122293 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:51.224699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58eee7a-15b7-4ba4-8b47-d2116e0c8051 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:51.232369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f4786-7949-41d0-9acf-fd9fea0252f4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:51.263079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca52c4c-ce33-4a50-8563-c0d5b52ec926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:51.264073 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca52c4c-ce33-4a50-8563-c0d5b52ec926 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:51.264553 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca52c4c-ce33-4a50-8563-c0d5b52ec926 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:51.265045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca52c4c-ce33-4a50-8563-c0d5b52ec926 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:51.265974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca52c4c-ce33-4a50-8563-c0d5b52ec926 None admin] RBAC: Adding query filter params (role.id=cae161e9c1b64656924448dd784ed4d1, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user.id=f0f33c461e67433593c4ca55f81f4e7e) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:51.266532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca52c4c-ce33-4a50-8563-c0d5b52ec926 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:51.268209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca52c4c-ce33-4a50-8563-c0d5b52ec926 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:51.274596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 88/177] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:51 2022] GET /identity/v3/role_assignments?user.id=f0f33c461e67433593c4ca55f81f4e7e&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=cae161e9c1b64656924448dd784ed4d1 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:51.276147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ce0cc-5a13-4c5a-921f-779c78e94dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:51.276539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcb002-4522-4323-9139-d887773cbd7b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:51.276762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ce0cc-5a13-4c5a-921f-779c78e94dea None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:51.277042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ce0cc-5a13-4c5a-921f-779c78e94dea None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:51.277311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ce0cc-5a13-4c5a-921f-779c78e94dea None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:51.277702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcb002-4522-4323-9139-d887773cbd7b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:51.277982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ce0cc-5a13-4c5a-921f-779c78e94dea None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:51.278320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ce0cc-5a13-4c5a-921f-779c78e94dea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:51.278654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcb002-4522-4323-9139-d887773cbd7b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:51.279784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ce0cc-5a13-4c5a-921f-779c78e94dea None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:51.280502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 89/178] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:51.288033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337d3e4-d6dc-4f00-b083-c217f2513f3b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:51.288648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337d3e4-d6dc-4f00-b083-c217f2513f3b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:51.289132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337d3e4-d6dc-4f00-b083-c217f2513f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:51.290746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 90/179] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:51.297309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ce0cc-5a13-4c5a-921f-779c78e94dea None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:51.311185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e337d3e4-d6dc-4f00-b083-c217f2513f3b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:51.312928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e337d3e4-d6dc-4f00-b083-c217f2513f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:51.366304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:51.416527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ce0cc-5a13-4c5a-921f-779c78e94dea None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:51.446602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:51.456795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6f4aa-f5cc-4bf6-b25f-7c489ede343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:51.458842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6f4aa-f5cc-4bf6-b25f-7c489ede343f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:51.458842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6f4aa-f5cc-4bf6-b25f-7c489ede343f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:51.458842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6f4aa-f5cc-4bf6-b25f-7c489ede343f None admin] PATH_INFO: `/v3/users/f0f33c461e67433593c4ca55f81f4e7e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:51.459325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6f4aa-f5cc-4bf6-b25f-7c489ede343f None admin] RBAC: Authorizing `identity:get_user(user_id=f0f33c461e67433593c4ca55f81f4e7e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:51.460812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6f4aa-f5cc-4bf6-b25f-7c489ede343f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:51.462682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 91/180] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:51 2022] GET /identity/v3/users/f0f33c461e67433593c4ca55f81f4e7e => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:51.463952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a5032-9f01-4590-8b4c-8a3d688be2e0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:51.464387 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a5032-9f01-4590-8b4c-8a3d688be2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:51.464486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a5032-9f01-4590-8b4c-8a3d688be2e0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:51.465731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 92/181] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:51.471089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a5032-9f01-4590-8b4c-8a3d688be2e0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:51.477041 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 90/182] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:51.478477 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633bae4-c8cf-4c3b-b70b-27e5e2a02723 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:51.478873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633bae4-c8cf-4c3b-b70b-27e5e2a02723 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:51.479240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633bae4-c8cf-4c3b-b70b-27e5e2a02723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:51.506682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4633bae4-c8cf-4c3b-b70b-27e5e2a02723 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:51.508157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4633bae4-c8cf-4c3b-b70b-27e5e2a02723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:51.590719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 91/183] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:51.591947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633bae4-c8cf-4c3b-b70b-27e5e2a02723 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:51.592249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a5032-9f01-4590-8b4c-8a3d688be2e0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:51.644925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d40130-e81a-45f3-b99a-a66eac28c912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:51.645576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d40130-e81a-45f3-b99a-a66eac28c912 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:51.645859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d40130-e81a-45f3-b99a-a66eac28c912 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:51.646153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d40130-e81a-45f3-b99a-a66eac28c912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:51.667305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d40130-e81a-45f3-b99a-a66eac28c912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d6d40130-e81a-45f3-b99a-a66eac28c912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:51.672239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:51.673086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:51.673858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 93/184] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:23:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:51.683518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d40130-e81a-45f3-b99a-a66eac28c912 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:51.735716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:51.754734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633bae4-c8cf-4c3b-b70b-27e5e2a02723 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:51.763867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:51.814574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006a317-953d-448f-b80f-54297567fdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:51.815310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006a317-953d-448f-b80f-54297567fdfa None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:51.815589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006a317-953d-448f-b80f-54297567fdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:51.816058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006a317-953d-448f-b80f-54297567fdfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:51.823883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d40130-e81a-45f3-b99a-a66eac28c912 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:51.867272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac9377-306e-412a-8c7c-f5fcc606b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:51.867944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac9377-306e-412a-8c7c-f5fcc606b1db None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:51.868222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac9377-306e-412a-8c7c-f5fcc606b1db None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:51.868499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac9377-306e-412a-8c7c-f5fcc606b1db None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:51.869263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac9377-306e-412a-8c7c-f5fcc606b1db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:51.869564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac9377-306e-412a-8c7c-f5fcc606b1db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:51.871150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac9377-306e-412a-8c7c-f5fcc606b1db None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:51.888367 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 94/185] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:51.888367 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac9377-306e-412a-8c7c-f5fcc606b1db None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:51.911098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006a317-953d-448f-b80f-54297567fdfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:51.912766 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006a317-953d-448f-b80f-54297567fdfa None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:52.000077 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac9377-306e-412a-8c7c-f5fcc606b1db None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:52.014265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:52.016792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:52.030655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 92/186] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:23:52.030655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006a317-953d-448f-b80f-54297567fdfa None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:52.041353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633dba-741a-464d-a5a4-a1dd68d79540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:52.042070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633dba-741a-464d-a5a4-a1dd68d79540 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:52.042375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633dba-741a-464d-a5a4-a1dd68d79540 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:52.042647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4633dba-741a-464d-a5a4-a1dd68d79540 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:52.053099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4633dba-741a-464d-a5a4-a1dd68d79540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d4633dba-741a-464d-a5a4-a1dd68d79540 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:52.057971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:52.059145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:23:52.060116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:52.063791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 95/187] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:23:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:52.064586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633dba-741a-464d-a5a4-a1dd68d79540 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:52.135107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006a317-953d-448f-b80f-54297567fdfa None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:52.170317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4633dba-741a-464d-a5a4-a1dd68d79540 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:52.172522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee70d0f-fbe0-4a01-9361-64cf7146ae0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:52.173146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee70d0f-fbe0-4a01-9361-64cf7146ae0e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:52.173421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee70d0f-fbe0-4a01-9361-64cf7146ae0e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:52.173687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee70d0f-fbe0-4a01-9361-64cf7146ae0e None admin] PATH_INFO: `/v3/users/d30c81fed7e04452bf3b501a0778eefd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:52.186619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee70d0f-fbe0-4a01-9361-64cf7146ae0e None admin] RBAC: Authorizing `identity:get_user(user_id=d30c81fed7e04452bf3b501a0778eefd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:52.188054 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee70d0f-fbe0-4a01-9361-64cf7146ae0e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:52.189720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 93/188] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:52 2022] GET /identity/v3/users/d30c81fed7e04452bf3b501a0778eefd => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:52.195048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee70d0f-fbe0-4a01-9361-64cf7146ae0e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:52.210521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf34e9-e29b-4918-ab88-4ea4b71a65e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:52.211229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf34e9-e29b-4918-ab88-4ea4b71a65e4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:52.211576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf34e9-e29b-4918-ab88-4ea4b71a65e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:52.211916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf34e9-e29b-4918-ab88-4ea4b71a65e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:52.213178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf34e9-e29b-4918-ab88-4ea4b71a65e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:52.214522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf34e9-e29b-4918-ab88-4ea4b71a65e4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:52.216350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 96/189] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:52.217104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf34e9-e29b-4918-ab88-4ea4b71a65e4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:52.247636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:52.262422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:52.302314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee70d0f-fbe0-4a01-9361-64cf7146ae0e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:52.320365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf34e9-e29b-4918-ab88-4ea4b71a65e4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:52.338921 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf40e6-04d9-4308-851a-f6b756b6232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:52.339840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf40e6-04d9-4308-851a-f6b756b6232a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:52.340115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf40e6-04d9-4308-851a-f6b756b6232a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:52.340388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf40e6-04d9-4308-851a-f6b756b6232a None admin] PATH_INFO: `/v3/projects/c13bf11626c248379284b9d7e67527bc` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:52.341504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf40e6-04d9-4308-851a-f6b756b6232a None admin] RBAC: Authorizing `identity:get_project(project_id=c13bf11626c248379284b9d7e67527bc)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:52.343391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf40e6-04d9-4308-851a-f6b756b6232a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:52.350363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 94/190] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:52 2022] GET /identity/v3/projects/c13bf11626c248379284b9d7e67527bc => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:52.350363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf40e6-04d9-4308-851a-f6b756b6232a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:52.362234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c48ce-b422-4b2e-a76a-0a004b3c319c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:52.363356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c48ce-b422-4b2e-a76a-0a004b3c319c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:52.363791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c48ce-b422-4b2e-a76a-0a004b3c319c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:52.364061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c48ce-b422-4b2e-a76a-0a004b3c319c None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:52.365022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c48ce-b422-4b2e-a76a-0a004b3c319c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:52.365323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c48ce-b422-4b2e-a76a-0a004b3c319c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:52.366768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c48ce-b422-4b2e-a76a-0a004b3c319c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:52.376139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 97/191] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:23:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:52.382329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c48ce-b422-4b2e-a76a-0a004b3c319c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:52.446719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf40e6-04d9-4308-851a-f6b756b6232a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:52.464020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:52.481614 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b58d7-41cc-4f0f-8262-d1e5907a096b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:52.482248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b58d7-41cc-4f0f-8262-d1e5907a096b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:52.482524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b58d7-41cc-4f0f-8262-d1e5907a096b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:52.482795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b58d7-41cc-4f0f-8262-d1e5907a096b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:52.491781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c48ce-b422-4b2e-a76a-0a004b3c319c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:52.494118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:52.526917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeaf993-1b5e-4702-8f05-cc4cb27bc7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:52.527448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaf993-1b5e-4702-8f05-cc4cb27bc7f7 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:52.527723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaf993-1b5e-4702-8f05-cc4cb27bc7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:52.528002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaf993-1b5e-4702-8f05-cc4cb27bc7f7 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:52.528647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeaf993-1b5e-4702-8f05-cc4cb27bc7f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:52.530947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeaf993-1b5e-4702-8f05-cc4cb27bc7f7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:52.553581 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b58d7-41cc-4f0f-8262-d1e5907a096b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:52.555058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b58d7-41cc-4f0f-8262-d1e5907a096b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:52.632580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 98/192] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:23:52 2022] POST /identity/v3/users => generated 297 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:23:52.633525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeaf993-1b5e-4702-8f05-cc4cb27bc7f7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:52.674177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 95/193] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:23:52.674177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b58d7-41cc-4f0f-8262-d1e5907a096b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:52.697640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:52.776066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeaf993-1b5e-4702-8f05-cc4cb27bc7f7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:52.812968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b58d7-41cc-4f0f-8262-d1e5907a096b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:52.821029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059e5a3-5b92-4ae6-ba60-5ae9d907624e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:52.821799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059e5a3-5b92-4ae6-ba60-5ae9d907624e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:52.822212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059e5a3-5b92-4ae6-ba60-5ae9d907624e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:52.822519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059e5a3-5b92-4ae6-ba60-5ae9d907624e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:52.823258 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059e5a3-5b92-4ae6-ba60-5ae9d907624e None admin] RBAC: Adding query filter params (role.id=1e04fdc0cc334fd7a733a1490a194e37, scope.project.id=c13bf11626c248379284b9d7e67527bc, user.id=f0f33c461e67433593c4ca55f81f4e7e) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:52.823565 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059e5a3-5b92-4ae6-ba60-5ae9d907624e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:52.826334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059e5a3-5b92-4ae6-ba60-5ae9d907624e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:52.834041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 99/194] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:52 2022] GET /identity/v3/role_assignments?user.id=f0f33c461e67433593c4ca55f81f4e7e&scope.project.id=c13bf11626c248379284b9d7e67527bc&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:52.834885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059e5a3-5b92-4ae6-ba60-5ae9d907624e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:52.864837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445d33b-f0cf-4cd2-bde3-857195f265bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:52.866122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445d33b-f0cf-4cd2-bde3-857195f265bd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:52.867294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:52.867564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445d33b-f0cf-4cd2-bde3-857195f265bd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:52.868535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445d33b-f0cf-4cd2-bde3-857195f265bd None admin] PATH_INFO: `/v3/users/d30c81fed7e04452bf3b501a0778eefd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:52.870204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445d33b-f0cf-4cd2-bde3-857195f265bd None admin] RBAC: Authorizing `identity:get_user(user_id=d30c81fed7e04452bf3b501a0778eefd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:52.872601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445d33b-f0cf-4cd2-bde3-857195f265bd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:52.875857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 96/195] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:52 2022] GET /identity/v3/users/d30c81fed7e04452bf3b501a0778eefd => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:52.882947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445d33b-f0cf-4cd2-bde3-857195f265bd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:52.980357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059e5a3-5b92-4ae6-ba60-5ae9d907624e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:53.024247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:53.028859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6dcc8-c1e9-451b-8ecf-84cc72126215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:53.029583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6dcc8-c1e9-451b-8ecf-84cc72126215 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:53.029939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6dcc8-c1e9-451b-8ecf-84cc72126215 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:53.030358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6dcc8-c1e9-451b-8ecf-84cc72126215 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:53.034214 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6dcc8-c1e9-451b-8ecf-84cc72126215 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:53.034214 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6dcc8-c1e9-451b-8ecf-84cc72126215 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:53.038132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 100/196] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:52 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:53.043101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6dcc8-c1e9-451b-8ecf-84cc72126215 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:53.050904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445d33b-f0cf-4cd2-bde3-857195f265bd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:53.102973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43065b9e-d928-47c8-a053-73c9f0727bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:53.103845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43065b9e-d928-47c8-a053-73c9f0727bdf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:53.104125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43065b9e-d928-47c8-a053-73c9f0727bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:53.104393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43065b9e-d928-47c8-a053-73c9f0727bdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:53.171778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:53.181343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6dcc8-c1e9-451b-8ecf-84cc72126215 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:53.185474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43065b9e-d928-47c8-a053-73c9f0727bdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:53.186980 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43065b9e-d928-47c8-a053-73c9f0727bdf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:53.221275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fae80-3820-47f2-b2e6-5fd733c16db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:53.222888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fae80-3820-47f2-b2e6-5fd733c16db7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:53.223449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fae80-3820-47f2-b2e6-5fd733c16db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:53.223901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fae80-3820-47f2-b2e6-5fd733c16db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:53.224809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fae80-3820-47f2-b2e6-5fd733c16db7 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=ce294031877d49d18dd64a30dbcc7c63, user.id=d30c81fed7e04452bf3b501a0778eefd) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:53.225320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fae80-3820-47f2-b2e6-5fd733c16db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:53.227202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fae80-3820-47f2-b2e6-5fd733c16db7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:53.233647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 101/197] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:53 2022] GET /identity/v3/role_assignments?user.id=d30c81fed7e04452bf3b501a0778eefd&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:53.285759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:53.305612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 97/198] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:23:53.310859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fae80-3820-47f2-b2e6-5fd733c16db7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:53.435488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fae80-3820-47f2-b2e6-5fd733c16db7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:53.471059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4a99c-5089-40fd-9f8e-0decb0c0f931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:53.471698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4a99c-5089-40fd-9f8e-0decb0c0f931 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:53.472552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4a99c-5089-40fd-9f8e-0decb0c0f931 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:53.472835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4a99c-5089-40fd-9f8e-0decb0c0f931 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:53.473985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4a99c-5089-40fd-9f8e-0decb0c0f931 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:53.475470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4a99c-5089-40fd-9f8e-0decb0c0f931 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:53.477118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 102/199] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:53 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:53.483024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43065b9e-d928-47c8-a053-73c9f0727bdf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:53.524681 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de014196-b2d9-4556-9f63-63c52a7d8add None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:53.525008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de014196-b2d9-4556-9f63-63c52a7d8add None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:53.525217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de014196-b2d9-4556-9f63-63c52a7d8add None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:53.526589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 103/200] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:23:53.531818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dc9e9-0f20-4dcf-8e28-d9e980a24ee0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:53.532099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dc9e9-0f20-4dcf-8e28-d9e980a24ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:53.532369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dc9e9-0f20-4dcf-8e28-d9e980a24ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:53.558366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-564dc9e9-0f20-4dcf-8e28-d9e980a24ee0 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:53.560379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-564dc9e9-0f20-4dcf-8e28-d9e980a24ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:53.603249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:53.659017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43065b9e-d928-47c8-a053-73c9f0727bdf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:53.668353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 104/201] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 04 05:23:53.715852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643efc24-0c00-4d95-acc2-9d7241c61781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:53.716808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643efc24-0c00-4d95-acc2-9d7241c61781 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:53.717282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643efc24-0c00-4d95-acc2-9d7241c61781 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:53.717757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643efc24-0c00-4d95-acc2-9d7241c61781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:53.718792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643efc24-0c00-4d95-acc2-9d7241c61781 None admin] RBAC: Adding query filter params (role.id=cae161e9c1b64656924448dd784ed4d1, scope.project.id=ce294031877d49d18dd64a30dbcc7c63, user.id=d30c81fed7e04452bf3b501a0778eefd) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:53.719299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643efc24-0c00-4d95-acc2-9d7241c61781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:53.720924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643efc24-0c00-4d95-acc2-9d7241c61781 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:53.730483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 98/202] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:53 2022] GET /identity/v3/role_assignments?user.id=d30c81fed7e04452bf3b501a0778eefd&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=cae161e9c1b64656924448dd784ed4d1 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:54.008472 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dc9e9-0f20-4dcf-8e28-d9e980a24ee0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:54.114965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dc9e9-0f20-4dcf-8e28-d9e980a24ee0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:54.154234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a2986c-3b5c-403a-b725-bdff06fce9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:54.155196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2986c-3b5c-403a-b725-bdff06fce9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:54.155667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2986c-3b5c-403a-b725-bdff06fce9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:54.156157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2986c-3b5c-403a-b725-bdff06fce9f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:54.166886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a2986c-3b5c-403a-b725-bdff06fce9f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-85a2986c-3b5c-403a-b725-bdff06fce9f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:54.171977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:54.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:23:54.173628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:54.176437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 105/203] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:54.182649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643efc24-0c00-4d95-acc2-9d7241c61781 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:54.251940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:54.318575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643efc24-0c00-4d95-acc2-9d7241c61781 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:54.358661 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8a601-0e27-4f1d-b619-33ab5786e371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:54.359630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a601-0e27-4f1d-b619-33ab5786e371 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:54.360124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a601-0e27-4f1d-b619-33ab5786e371 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:54.360587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a601-0e27-4f1d-b619-33ab5786e371 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:54.361490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8a601-0e27-4f1d-b619-33ab5786e371 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:54.361982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8a601-0e27-4f1d-b619-33ab5786e371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:54.364204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8a601-0e27-4f1d-b619-33ab5786e371 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:54.372382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 99/204] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:54.378183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a2986c-3b5c-403a-b725-bdff06fce9f3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:54.396282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:54.482459 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a2986c-3b5c-403a-b725-bdff06fce9f3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:54.517616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233a0b4-a664-4f03-96f4-cd526a6881e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:54.518721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233a0b4-a664-4f03-96f4-cd526a6881e4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:54.519221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233a0b4-a664-4f03-96f4-cd526a6881e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:54.519687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233a0b4-a664-4f03-96f4-cd526a6881e4 None admin] PATH_INFO: `/v3/users/e7f18540e5e841e580c1915e9309de5c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:54.521006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233a0b4-a664-4f03-96f4-cd526a6881e4 None admin] RBAC: Authorizing `identity:get_user(user_id=e7f18540e5e841e580c1915e9309de5c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:54.522832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233a0b4-a664-4f03-96f4-cd526a6881e4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:54.524617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 106/205] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:54 2022] GET /identity/v3/users/e7f18540e5e841e580c1915e9309de5c => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:54.530257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8a601-0e27-4f1d-b619-33ab5786e371 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:54.655014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8a601-0e27-4f1d-b619-33ab5786e371 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:54.671651 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:54.699062 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72484f7-6f88-4dbf-b017-d7d442e86c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:54.700031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72484f7-6f88-4dbf-b017-d7d442e86c2e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:54.700526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72484f7-6f88-4dbf-b017-d7d442e86c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:54.700991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72484f7-6f88-4dbf-b017-d7d442e86c2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:54.777408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72484f7-6f88-4dbf-b017-d7d442e86c2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:54.779245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72484f7-6f88-4dbf-b017-d7d442e86c2e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:54.904089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 100/206] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:23:54.910867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233a0b4-a664-4f03-96f4-cd526a6881e4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:54.994784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:55.013602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233a0b4-a664-4f03-96f4-cd526a6881e4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:55.055292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd94bf5-dd6c-4d42-b756-d015a7b1c864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:55.055937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd94bf5-dd6c-4d42-b756-d015a7b1c864 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:55.056218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd94bf5-dd6c-4d42-b756-d015a7b1c864 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:55.056491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd94bf5-dd6c-4d42-b756-d015a7b1c864 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:55.062851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd94bf5-dd6c-4d42-b756-d015a7b1c864 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:55.064311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd94bf5-dd6c-4d42-b756-d015a7b1c864 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:55.066534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 107/207] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:54 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:55.072818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72484f7-6f88-4dbf-b017-d7d442e86c2e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:55.093286 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:55.237806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72484f7-6f88-4dbf-b017-d7d442e86c2e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:55.280670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fbcfa-fcd3-4169-821e-86efc8f39574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:55.281615 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fbcfa-fcd3-4169-821e-86efc8f39574 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:55.282183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fbcfa-fcd3-4169-821e-86efc8f39574 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:55.282649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fbcfa-fcd3-4169-821e-86efc8f39574 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:55.283563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fbcfa-fcd3-4169-821e-86efc8f39574 None admin] RBAC: Adding query filter params (role.id=840afb6280c341c6beefc211b040c463, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9, user.id=e7f18540e5e841e580c1915e9309de5c) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:55.284074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fbcfa-fcd3-4169-821e-86efc8f39574 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:55.285731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fbcfa-fcd3-4169-821e-86efc8f39574 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:55.296157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 101/208] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:55 2022] GET /identity/v3/role_assignments?user.id=e7f18540e5e841e580c1915e9309de5c&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=840afb6280c341c6beefc211b040c463 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:55.710232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffdc22-4fea-47d6-9e15-dafcc6068d4b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:55.711253 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffdc22-4fea-47d6-9e15-dafcc6068d4b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:55.711538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffdc22-4fea-47d6-9e15-dafcc6068d4b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:55.713075 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 108/209] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:55.718841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29920db2-7b54-41d9-b90d-0e8481a76f5b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:55.719132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29920db2-7b54-41d9-b90d-0e8481a76f5b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:55.719413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29920db2-7b54-41d9-b90d-0e8481a76f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:55.738648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-29920db2-7b54-41d9-b90d-0e8481a76f5b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:55.740132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-29920db2-7b54-41d9-b90d-0e8481a76f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:55.787751 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c5970-1c4c-4605-a66b-ac3056bb3364 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:55.788365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c5970-1c4c-4605-a66b-ac3056bb3364 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:55.788863 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c5970-1c4c-4605-a66b-ac3056bb3364 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:55.790495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 109/210] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:55.795713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95843329-309d-430f-a6c9-fdd6e27d5b5b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:55.796265 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95843329-309d-430f-a6c9-fdd6e27d5b5b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:55.796761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95843329-309d-430f-a6c9-fdd6e27d5b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:55.823896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-95843329-309d-430f-a6c9-fdd6e27d5b5b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:55.825425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-95843329-309d-430f-a6c9-fdd6e27d5b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:55.827251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 102/211] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:55.905972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 110/212] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:23:56.110824 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95843329-309d-430f-a6c9-fdd6e27d5b5b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:56.111007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362605d4-a6cf-4211-b19b-bda10df52579 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.111245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362605d4-a6cf-4211-b19b-bda10df52579 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.111704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362605d4-a6cf-4211-b19b-bda10df52579 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.114044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 103/213] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:56.119103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528c122-9675-4b29-b81e-bdabf9d2a2e3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.119555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528c122-9675-4b29-b81e-bdabf9d2a2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.119969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528c122-9675-4b29-b81e-bdabf9d2a2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.128242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:56.139035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6528c122-9675-4b29-b81e-bdabf9d2a2e3 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:56.140498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6528c122-9675-4b29-b81e-bdabf9d2a2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:56.200344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:56.211746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 104/214] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:23:56.212404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528c122-9675-4b29-b81e-bdabf9d2a2e3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:56.231257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95843329-309d-430f-a6c9-fdd6e27d5b5b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:56.267055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b3a18-9519-456f-a79c-df44a4497590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:56.268037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b3a18-9519-456f-a79c-df44a4497590 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.268521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b3a18-9519-456f-a79c-df44a4497590 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.269481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b3a18-9519-456f-a79c-df44a4497590 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.280740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b3a18-9519-456f-a79c-df44a4497590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-366b3a18-9519-456f-a79c-df44a4497590 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:56.287069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:56.288248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:23:56.289269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:56.291574 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 111/215] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:23:56.297893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b3a18-9519-456f-a79c-df44a4497590 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:56.343211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528c122-9675-4b29-b81e-bdabf9d2a2e3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:56.397737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd8db9-d645-4773-9830-9e441af3cec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:56.398418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd8db9-d645-4773-9830-9e441af3cec1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.398689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd8db9-d645-4773-9830-9e441af3cec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.398951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd8db9-d645-4773-9830-9e441af3cec1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.400089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd8db9-d645-4773-9830-9e441af3cec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:56.401537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd8db9-d645-4773-9830-9e441af3cec1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:56.403574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 105/216] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:56.410185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e16d1b-bd8e-4be8-8d56-e3973418c6dd None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.410185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e16d1b-bd8e-4be8-8d56-e3973418c6dd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.410521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e16d1b-bd8e-4be8-8d56-e3973418c6dd None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.411839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 106/217] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:56.412375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e16d1b-bd8e-4be8-8d56-e3973418c6dd None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:56.417576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:56.418362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b3a18-9519-456f-a79c-df44a4497590 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:56.454739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315739a-d822-4e96-b970-8bac1c0e5af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:56.454942 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315739a-d822-4e96-b970-8bac1c0e5af9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.455145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315739a-d822-4e96-b970-8bac1c0e5af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.455435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315739a-d822-4e96-b970-8bac1c0e5af9 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.456615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315739a-d822-4e96-b970-8bac1c0e5af9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:56.456923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315739a-d822-4e96-b970-8bac1c0e5af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:56.458320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315739a-d822-4e96-b970-8bac1c0e5af9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:56.467026 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 112/218] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:56.468175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cef6a-bda2-49ab-8a04-2896af5ec14f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.468466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cef6a-bda2-49ab-8a04-2896af5ec14f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.468751 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cef6a-bda2-49ab-8a04-2896af5ec14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.475685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:56.489387 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-816cef6a-bda2-49ab-8a04-2896af5ec14f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:56.490611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-816cef6a-bda2-49ab-8a04-2896af5ec14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:56.523315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e16d1b-bd8e-4be8-8d56-e3973418c6dd None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:56.555656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5ee0f-3e13-4031-a959-30051b2cbefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:56.556267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ee0f-3e13-4031-a959-30051b2cbefb None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.556544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ee0f-3e13-4031-a959-30051b2cbefb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.556820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5ee0f-3e13-4031-a959-30051b2cbefb None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.556943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 113/219] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:23:56.557465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5ee0f-3e13-4031-a959-30051b2cbefb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:56.557649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cef6a-bda2-49ab-8a04-2896af5ec14f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:56.558941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5ee0f-3e13-4031-a959-30051b2cbefb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:56.655608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cef6a-bda2-49ab-8a04-2896af5ec14f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:56.669086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 107/220] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:23:56 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:23:56.669909 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5ee0f-3e13-4031-a959-30051b2cbefb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:56.709245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:56.719055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f508cc4-db10-451b-9c21-eda2d221a6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:56.721507 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f508cc4-db10-451b-9c21-eda2d221a6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.722020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f508cc4-db10-451b-9c21-eda2d221a6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.722465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f508cc4-db10-451b-9c21-eda2d221a6f8 None admin] PATH_INFO: `/v3/users/1417f9fa5c2c4c2dbc09109785b66b29` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.745608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f508cc4-db10-451b-9c21-eda2d221a6f8 None admin] RBAC: Authorizing `identity:get_user(user_id=1417f9fa5c2c4c2dbc09109785b66b29)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:56.747890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f508cc4-db10-451b-9c21-eda2d221a6f8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:56.750603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 114/221] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:56 2022] GET /identity/v3/users/1417f9fa5c2c4c2dbc09109785b66b29 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:56.752180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:56.760705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f508cc4-db10-451b-9c21-eda2d221a6f8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:56.802310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5ee0f-3e13-4031-a959-30051b2cbefb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:56.847585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e84b0-1d29-4283-aa2a-4277668e5342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:56.848560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e84b0-1d29-4283-aa2a-4277668e5342 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.848958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e84b0-1d29-4283-aa2a-4277668e5342 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.849227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e84b0-1d29-4283-aa2a-4277668e5342 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.862987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e84b0-1d29-4283-aa2a-4277668e5342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3e4e84b0-1d29-4283-aa2a-4277668e5342 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:56.868361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:56.869306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:23:56.870196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:56.874419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 108/222] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:23:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:56.875283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e84b0-1d29-4283-aa2a-4277668e5342 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:56.879949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f508cc4-db10-451b-9c21-eda2d221a6f8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:56.924182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6598b-c5a7-49a2-88f7-4aad8fffd8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:56.924803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6598b-c5a7-49a2-88f7-4aad8fffd8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:56.925643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6598b-c5a7-49a2-88f7-4aad8fffd8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:56.925911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6598b-c5a7-49a2-88f7-4aad8fffd8b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:56.998328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:57.006639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6598b-c5a7-49a2-88f7-4aad8fffd8b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:57.008146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6598b-c5a7-49a2-88f7-4aad8fffd8b5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:57.054080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e84b0-1d29-4283-aa2a-4277668e5342 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:57.072238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:57.089622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc9a64-b114-4475-8fb1-242a95c8c29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:57.093832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc9a64-b114-4475-8fb1-242a95c8c29b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:57.093832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc9a64-b114-4475-8fb1-242a95c8c29b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:57.093832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc9a64-b114-4475-8fb1-242a95c8c29b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:57.102174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc9a64-b114-4475-8fb1-242a95c8c29b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-24fc9a64-b114-4475-8fb1-242a95c8c29b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:57.107255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:57.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:23:57.108978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 109/223] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:23:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:57.109627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc9a64-b114-4475-8fb1-242a95c8c29b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:57.133815 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 115/224] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:23:57.134706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6598b-c5a7-49a2-88f7-4aad8fffd8b5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:57.215362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc9a64-b114-4475-8fb1-242a95c8c29b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:57.251449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390a376-3dc2-4fc7-9314-6302a0fb7228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:57.252096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a376-3dc2-4fc7-9314-6302a0fb7228 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:57.252418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a376-3dc2-4fc7-9314-6302a0fb7228 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:57.252886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:57.254443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390a376-3dc2-4fc7-9314-6302a0fb7228 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:57.255314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390a376-3dc2-4fc7-9314-6302a0fb7228 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:57.255781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390a376-3dc2-4fc7-9314-6302a0fb7228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:57.257320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390a376-3dc2-4fc7-9314-6302a0fb7228 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:57.264499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6598b-c5a7-49a2-88f7-4aad8fffd8b5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:57.266479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 110/225] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:57.267433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390a376-3dc2-4fc7-9314-6302a0fb7228 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:57.309128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2893bb7-9588-4eb3-9222-ba530391ffba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:57.309747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2893bb7-9588-4eb3-9222-ba530391ffba None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:57.310020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2893bb7-9588-4eb3-9222-ba530391ffba None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:57.310318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2893bb7-9588-4eb3-9222-ba530391ffba None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:57.310989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2893bb7-9588-4eb3-9222-ba530391ffba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:57.311293 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2893bb7-9588-4eb3-9222-ba530391ffba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:57.313212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2893bb7-9588-4eb3-9222-ba530391ffba None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:57.315347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:57.325344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 116/226] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:23:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:57.326197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2893bb7-9588-4eb3-9222-ba530391ffba None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:57.379648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390a376-3dc2-4fc7-9314-6302a0fb7228 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:57.421749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a8682-7455-4f25-964f-e3672dbb86db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:57.422423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a8682-7455-4f25-964f-e3672dbb86db None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:57.422700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a8682-7455-4f25-964f-e3672dbb86db None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:57.422970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a8682-7455-4f25-964f-e3672dbb86db None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:57.424148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a8682-7455-4f25-964f-e3672dbb86db None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:57.426677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a8682-7455-4f25-964f-e3672dbb86db None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:57.429387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 111/227] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:57 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:57.430290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a8682-7455-4f25-964f-e3672dbb86db None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:57.456192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2893bb7-9588-4eb3-9222-ba530391ffba None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:57.495452 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cf4f1-8c1d-478c-ad01-500ead127f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:57.496088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cf4f1-8c1d-478c-ad01-500ead127f5d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:57.496280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cf4f1-8c1d-478c-ad01-500ead127f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:57.496555 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cf4f1-8c1d-478c-ad01-500ead127f5d None admin] PATH_INFO: `/v3/users/d30c81fed7e04452bf3b501a0778eefd` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:57.497754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cf4f1-8c1d-478c-ad01-500ead127f5d None admin] RBAC: Authorizing `identity:get_user(user_id=d30c81fed7e04452bf3b501a0778eefd)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:57.499158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cf4f1-8c1d-478c-ad01-500ead127f5d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:57.501037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 117/228] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:57 2022] GET /identity/v3/users/d30c81fed7e04452bf3b501a0778eefd => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:57.501906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cf4f1-8c1d-478c-ad01-500ead127f5d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:57.529938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:57.558101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a8682-7455-4f25-964f-e3672dbb86db None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:57.569050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:57.594284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e56847-ced6-444f-8c4e-1f8d50e39a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:57.594688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e56847-ced6-444f-8c4e-1f8d50e39a27 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:57.594969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e56847-ced6-444f-8c4e-1f8d50e39a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:57.595289 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e56847-ced6-444f-8c4e-1f8d50e39a27 None admin] PATH_INFO: `/v3/users/d30c81fed7e04452bf3b501a0778eefd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:57.596429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e56847-ced6-444f-8c4e-1f8d50e39a27 None admin] RBAC: Authorizing `identity:get_user(user_id=d30c81fed7e04452bf3b501a0778eefd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:57.598491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e56847-ced6-444f-8c4e-1f8d50e39a27 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:57.605278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 112/229] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:23:57 2022] GET /identity/v3/users/d30c81fed7e04452bf3b501a0778eefd => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:57.606234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e56847-ced6-444f-8c4e-1f8d50e39a27 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:57.637553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cf4f1-8c1d-478c-ad01-500ead127f5d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:57.684413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204b179-99d4-47e6-b5d3-ead859eb896f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:57.685441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204b179-99d4-47e6-b5d3-ead859eb896f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:57.685896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204b179-99d4-47e6-b5d3-ead859eb896f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:57.686360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204b179-99d4-47e6-b5d3-ead859eb896f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:57.687269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204b179-99d4-47e6-b5d3-ead859eb896f None admin] RBAC: Adding query filter params (role.id=1e04fdc0cc334fd7a733a1490a194e37, scope.project.id=ce294031877d49d18dd64a30dbcc7c63, user.id=1417f9fa5c2c4c2dbc09109785b66b29) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:23:57.687757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204b179-99d4-47e6-b5d3-ead859eb896f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:57.689876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204b179-99d4-47e6-b5d3-ead859eb896f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:57.697849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 118/230] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:23:57 2022] GET /identity/v3/role_assignments?user.id=1417f9fa5c2c4c2dbc09109785b66b29&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:57.699223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204b179-99d4-47e6-b5d3-ead859eb896f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:57.726917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e56847-ced6-444f-8c4e-1f8d50e39a27 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:57.796997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef97ef-4dbc-4e6b-9cd7-b4c531b5eb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:57.798024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef97ef-4dbc-4e6b-9cd7-b4c531b5eb32 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:57.798332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef97ef-4dbc-4e6b-9cd7-b4c531b5eb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:57.798607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef97ef-4dbc-4e6b-9cd7-b4c531b5eb32 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:57.802850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef97ef-4dbc-4e6b-9cd7-b4c531b5eb32 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:57.804712 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef97ef-4dbc-4e6b-9cd7-b4c531b5eb32 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:57.808288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 113/231] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:57 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:57.812263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef97ef-4dbc-4e6b-9cd7-b4c531b5eb32 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:57.878924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:57.913211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204b179-99d4-47e6-b5d3-ead859eb896f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:57.919469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:57.973204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389cc8f-15e5-4a09-8369-01ab09cbf810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:57.974726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389cc8f-15e5-4a09-8369-01ab09cbf810 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:57.975206 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389cc8f-15e5-4a09-8369-01ab09cbf810 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:57.976022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389cc8f-15e5-4a09-8369-01ab09cbf810 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:57.977416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389cc8f-15e5-4a09-8369-01ab09cbf810 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:57.979590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389cc8f-15e5-4a09-8369-01ab09cbf810 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:57.981494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 119/232] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:23:57 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:23:57.987832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389cc8f-15e5-4a09-8369-01ab09cbf810 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:58.022129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef97ef-4dbc-4e6b-9cd7-b4c531b5eb32 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:58.086485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c1100-6996-4975-9198-0f8c599b5d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:58.087512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c1100-6996-4975-9198-0f8c599b5d79 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:58.088940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c1100-6996-4975-9198-0f8c599b5d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:58.089416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c1100-6996-4975-9198-0f8c599b5d79 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63/users/d30c81fed7e04452bf3b501a0778eefd/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:58.092939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c1100-6996-4975-9198-0f8c599b5d79 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce294031877d49d18dd64a30dbcc7c63, user_id=d30c81fed7e04452bf3b501a0778eefd, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:58.095999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c1100-6996-4975-9198-0f8c599b5d79 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:58.145339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389cc8f-15e5-4a09-8369-01ab09cbf810 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:58.184000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 114/233] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:23:57 2022] PUT /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63/users/d30c81fed7e04452bf3b501a0778eefd/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:23:58.197598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab5184-d707-4b2f-adc7-89dac9e250e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:58.198715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab5184-d707-4b2f-adc7-89dac9e250e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:58.199210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab5184-d707-4b2f-adc7-89dac9e250e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:58.199657 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab5184-d707-4b2f-adc7-89dac9e250e6 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63/users/d30c81fed7e04452bf3b501a0778eefd/roles/cae161e9c1b64656924448dd784ed4d1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:58.204176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab5184-d707-4b2f-adc7-89dac9e250e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce294031877d49d18dd64a30dbcc7c63, user_id=d30c81fed7e04452bf3b501a0778eefd, role_id=cae161e9c1b64656924448dd784ed4d1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:58.205695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab5184-d707-4b2f-adc7-89dac9e250e6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:58.287277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 120/234] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:23:57 2022] PUT /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63/users/d30c81fed7e04452bf3b501a0778eefd/roles/cae161e9c1b64656924448dd784ed4d1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:23:59.137257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1efda9-3bea-41bc-b473-8cb2c2dd8dd7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:59.137766 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1efda9-3bea-41bc-b473-8cb2c2dd8dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:59.137766 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1efda9-3bea-41bc-b473-8cb2c2dd8dd7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:59.139429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 115/235] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:59.145380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af142198-1fa2-4f9f-9500-3f790dee62e2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:59.145792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af142198-1fa2-4f9f-9500-3f790dee62e2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:59.145864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af142198-1fa2-4f9f-9500-3f790dee62e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:59.164786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-af142198-1fa2-4f9f-9500-3f790dee62e2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:59.165964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-af142198-1fa2-4f9f-9500-3f790dee62e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:59.173581 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:59.231057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512138b-2040-4f0c-83e1-e6f4a966ed57 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:59.231377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512138b-2040-4f0c-83e1-e6f4a966ed57 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:59.231603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512138b-2040-4f0c-83e1-e6f4a966ed57 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:59.233106 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 116/236] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:23:59.238228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562de96-8d67-4263-8962-129c6aab6e44 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:59.238491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562de96-8d67-4263-8962-129c6aab6e44 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:59.238747 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562de96-8d67-4263-8962-129c6aab6e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:59.269957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 121/237] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:23:59.278467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0562de96-8d67-4263-8962-129c6aab6e44 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:23:59.279680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0562de96-8d67-4263-8962-129c6aab6e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:23:59.368007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 117/238] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:23:59.677098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af142198-1fa2-4f9f-9500-3f790dee62e2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:59.715601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562de96-8d67-4263-8962-129c6aab6e44 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:59.781565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:59.814737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af142198-1fa2-4f9f-9500-3f790dee62e2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:59.877285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88f506-7ab0-49a6-942f-4229dcfddf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:59.877971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88f506-7ab0-49a6-942f-4229dcfddf0d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:59.878775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88f506-7ab0-49a6-942f-4229dcfddf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:59.879060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88f506-7ab0-49a6-942f-4229dcfddf0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:59.905146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88f506-7ab0-49a6-942f-4229dcfddf0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7c88f506-7ab0-49a6-942f-4229dcfddf0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:23:59.911376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:23:59.912234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:23:59.913018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:23:59.915442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562de96-8d67-4263-8962-129c6aab6e44 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:23:59.915884 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 122/239] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:23:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:23:59.922273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88f506-7ab0-49a6-942f-4229dcfddf0d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:23:59.940084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:23:59.975106 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca257aa2-e103-451e-a19d-1de0315ff1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:23:59.975774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca257aa2-e103-451e-a19d-1de0315ff1fd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:23:59.976055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca257aa2-e103-451e-a19d-1de0315ff1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:23:59.976359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca257aa2-e103-451e-a19d-1de0315ff1fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:23:59.977643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca257aa2-e103-451e-a19d-1de0315ff1fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:23:59.979114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca257aa2-e103-451e-a19d-1de0315ff1fd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:23:59.981101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 118/240] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:23:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:23:59.988774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca257aa2-e103-451e-a19d-1de0315ff1fd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:00.117365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88f506-7ab0-49a6-942f-4229dcfddf0d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:00.157069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca257aa2-e103-451e-a19d-1de0315ff1fd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:00.158831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:00.188547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60477203-282b-4309-977d-e0d97498bdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:00.189205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60477203-282b-4309-977d-e0d97498bdaa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:00.189421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60477203-282b-4309-977d-e0d97498bdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:00.189691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60477203-282b-4309-977d-e0d97498bdaa None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:00.190477 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60477203-282b-4309-977d-e0d97498bdaa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:00.191314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60477203-282b-4309-977d-e0d97498bdaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:00.192684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60477203-282b-4309-977d-e0d97498bdaa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:00.207565 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 123/241] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:23:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:00.213153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60477203-282b-4309-977d-e0d97498bdaa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:00.217400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8618a19-fbf5-4b9d-a65b-ff8f07f4bed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:00.218012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8618a19-fbf5-4b9d-a65b-ff8f07f4bed1 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:00.218311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8618a19-fbf5-4b9d-a65b-ff8f07f4bed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:00.218588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8618a19-fbf5-4b9d-a65b-ff8f07f4bed1 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:00.219239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8618a19-fbf5-4b9d-a65b-ff8f07f4bed1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:00.220681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8618a19-fbf5-4b9d-a65b-ff8f07f4bed1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:00.306544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:00.331481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60477203-282b-4309-977d-e0d97498bdaa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:00.357540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 119/242] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:23:59 2022] POST /identity/v3/users => generated 293 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:24:00.373974 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b17e0-3523-4539-afd2-d5e8cf0425ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:00.374680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b17e0-3523-4539-afd2-d5e8cf0425ef None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:00.374961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b17e0-3523-4539-afd2-d5e8cf0425ef None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:00.375237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b17e0-3523-4539-afd2-d5e8cf0425ef None admin] PATH_INFO: `/v3/users/9b9d59b9940e419c9086fd47cfcf07ee` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:00.380508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a898f8d-aaed-40dc-ae8b-ec2ee8bb6f44 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:00.380815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a898f8d-aaed-40dc-ae8b-ec2ee8bb6f44 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:00.381056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a898f8d-aaed-40dc-ae8b-ec2ee8bb6f44 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:00.382574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 120/243] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:24:00.388373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24717a86-8861-4316-8ad6-8391ef963136 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:00.388673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24717a86-8861-4316-8ad6-8391ef963136 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:00.388926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24717a86-8861-4316-8ad6-8391ef963136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:00.397579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b17e0-3523-4539-afd2-d5e8cf0425ef None admin] RBAC: Authorizing `identity:get_user(user_id=9b9d59b9940e419c9086fd47cfcf07ee)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:00.399155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b17e0-3523-4539-afd2-d5e8cf0425ef None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:00.404434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 124/244] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:00 2022] GET /identity/v3/users/9b9d59b9940e419c9086fd47cfcf07ee => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:00.411124 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b17e0-3523-4539-afd2-d5e8cf0425ef None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:00.419763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-24717a86-8861-4316-8ad6-8391ef963136 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:00.426643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-24717a86-8861-4316-8ad6-8391ef963136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:00.564212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 121/245] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:00.593761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b17e0-3523-4539-afd2-d5e8cf0425ef None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:00.618485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:00.631592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b029cd2-985e-4b08-9392-99a3413b86dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:00.635310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b029cd2-985e-4b08-9392-99a3413b86dd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:00.635310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b029cd2-985e-4b08-9392-99a3413b86dd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:00.635310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b029cd2-985e-4b08-9392-99a3413b86dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:00.709033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b029cd2-985e-4b08-9392-99a3413b86dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:00.710528 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b029cd2-985e-4b08-9392-99a3413b86dd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:00.823713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 125/246] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:00.829653 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24717a86-8861-4316-8ad6-8391ef963136 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:00.851984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b029cd2-985e-4b08-9392-99a3413b86dd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:00.879674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:00.975888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:00.991439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24717a86-8861-4316-8ad6-8391ef963136 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:00.991648 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b029cd2-985e-4b08-9392-99a3413b86dd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:01.036375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471eb99d-22fa-485a-a338-62eefeaf542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:01.037030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471eb99d-22fa-485a-a338-62eefeaf542f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.037309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471eb99d-22fa-485a-a338-62eefeaf542f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.037581 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471eb99d-22fa-485a-a338-62eefeaf542f None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.038841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471eb99d-22fa-485a-a338-62eefeaf542f None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:01.039148 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c4eab-7c18-4fdb-8af5-bcf1cad57f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:01.039791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c4eab-7c18-4fdb-8af5-bcf1cad57f5c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.040132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c4eab-7c18-4fdb-8af5-bcf1cad57f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.040362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471eb99d-22fa-485a-a338-62eefeaf542f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:01.040500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c4eab-7c18-4fdb-8af5-bcf1cad57f5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.042242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 122/247] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:00 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:01.043747 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afaea6f-99fa-4d70-bbd4-85f3fa315525 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.044024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afaea6f-99fa-4d70-bbd4-85f3fa315525 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.044294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afaea6f-99fa-4d70-bbd4-85f3fa315525 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.047170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 123/248] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:01.048290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50365f06-d090-464a-ab98-844bd5b6b7d0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.048568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50365f06-d090-464a-ab98-844bd5b6b7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.048844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50365f06-d090-464a-ab98-844bd5b6b7d0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.050819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 124/249] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:01.051369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50365f06-d090-464a-ab98-844bd5b6b7d0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:01.064847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c4eab-7c18-4fdb-8af5-bcf1cad57f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-576c4eab-7c18-4fdb-8af5-bcf1cad57f5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:01.070797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:01.071888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:01.072876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:01.075085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 126/250] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:01.075749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92f924-32c0-47e8-a91d-77eab1abd6cb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.075866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92f924-32c0-47e8-a91d-77eab1abd6cb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.076204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92f924-32c0-47e8-a91d-77eab1abd6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.100949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0a92f924-32c0-47e8-a91d-77eab1abd6cb None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:01.102548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0a92f924-32c0-47e8-a91d-77eab1abd6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:01.166935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50365f06-d090-464a-ab98-844bd5b6b7d0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:01.191085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 127/251] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:01.192520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f416f9-e6de-4e6d-baae-de2ed61ef091 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.192812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f416f9-e6de-4e6d-baae-de2ed61ef091 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.193095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f416f9-e6de-4e6d-baae-de2ed61ef091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.207784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64b9a5-18a9-451c-80ab-8efb5cd59105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:01.208736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64b9a5-18a9-451c-80ab-8efb5cd59105 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.209249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64b9a5-18a9-451c-80ab-8efb5cd59105 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.209758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64b9a5-18a9-451c-80ab-8efb5cd59105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.210777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64b9a5-18a9-451c-80ab-8efb5cd59105 None admin] RBAC: Adding query filter params (role.id=840afb6280c341c6beefc211b040c463, scope.project.id=ce294031877d49d18dd64a30dbcc7c63, user.id=9b9d59b9940e419c9086fd47cfcf07ee) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:01.211300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64b9a5-18a9-451c-80ab-8efb5cd59105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:01.213087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64b9a5-18a9-451c-80ab-8efb5cd59105 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:01.214873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b5f416f9-e6de-4e6d-baae-de2ed61ef091 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:01.216257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b5f416f9-e6de-4e6d-baae-de2ed61ef091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:01.220010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 125/252] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:01 2022] GET /identity/v3/role_assignments?user.id=9b9d59b9940e419c9086fd47cfcf07ee&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=840afb6280c341c6beefc211b040c463 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:01.221072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64b9a5-18a9-451c-80ab-8efb5cd59105 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:01.227047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:01.247466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:01.320283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 128/253] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:01.368546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64b9a5-18a9-451c-80ab-8efb5cd59105 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:01.420191 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad842b-c547-4ebb-b1d6-b41c9d7e93e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:01.420826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad842b-c547-4ebb-b1d6-b41c9d7e93e0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.421109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad842b-c547-4ebb-b1d6-b41c9d7e93e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.421385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad842b-c547-4ebb-b1d6-b41c9d7e93e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.422074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad842b-c547-4ebb-b1d6-b41c9d7e93e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:01.422409 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad842b-c547-4ebb-b1d6-b41c9d7e93e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:01.423807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad842b-c547-4ebb-b1d6-b41c9d7e93e0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:01.437762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 126/254] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:01.443567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f416f9-e6de-4e6d-baae-de2ed61ef091 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:01.572114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f416f9-e6de-4e6d-baae-de2ed61ef091 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:01.586575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad842b-c547-4ebb-b1d6-b41c9d7e93e0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:01.613337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def7355-395b-434e-9157-1bd8226b43e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:01.614435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def7355-395b-434e-9157-1bd8226b43e5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.614928 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def7355-395b-434e-9157-1bd8226b43e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.615387 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def7355-395b-434e-9157-1bd8226b43e5 None admin] PATH_INFO: `/v3/users/1417f9fa5c2c4c2dbc09109785b66b29` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.616736 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def7355-395b-434e-9157-1bd8226b43e5 None admin] RBAC: Authorizing `identity:get_user(user_id=1417f9fa5c2c4c2dbc09109785b66b29)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:01.618542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def7355-395b-434e-9157-1bd8226b43e5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:01.620849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 129/255] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:01 2022] GET /identity/v3/users/1417f9fa5c2c4c2dbc09109785b66b29 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:01.628538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def7355-395b-434e-9157-1bd8226b43e5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:01.654542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:01.736493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:01.757731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad842b-c547-4ebb-b1d6-b41c9d7e93e0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:01.808108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f77e96-dc29-4f28-b973-2dfeba81c08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:01.809326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f77e96-dc29-4f28-b973-2dfeba81c08c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.810696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f77e96-dc29-4f28-b973-2dfeba81c08c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.811376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f77e96-dc29-4f28-b973-2dfeba81c08c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.824479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def7355-395b-434e-9157-1bd8226b43e5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:01.838739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f77e96-dc29-4f28-b973-2dfeba81c08c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a6f77e96-dc29-4f28-b973-2dfeba81c08c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:01.846218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:01.847124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:01.847884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:01.852611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 127/256] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:24:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:01.854853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f77e96-dc29-4f28-b973-2dfeba81c08c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:01.879116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3da3d3-48e8-4cf1-851d-a34511aac688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:01.880171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3da3d3-48e8-4cf1-851d-a34511aac688 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:01.880912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3da3d3-48e8-4cf1-851d-a34511aac688 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:01.881391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3da3d3-48e8-4cf1-851d-a34511aac688 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:01.883372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3da3d3-48e8-4cf1-851d-a34511aac688 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:01.894379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3da3d3-48e8-4cf1-851d-a34511aac688 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:01.896976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 130/257] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:01 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:01.897944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3da3d3-48e8-4cf1-851d-a34511aac688 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:02.002592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:02.012348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3da3d3-48e8-4cf1-851d-a34511aac688 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:02.056376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993d8d6-8471-4703-85e4-ee844f411152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:02.057501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993d8d6-8471-4703-85e4-ee844f411152 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:02.060467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993d8d6-8471-4703-85e4-ee844f411152 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:02.060942 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993d8d6-8471-4703-85e4-ee844f411152 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:02.061823 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993d8d6-8471-4703-85e4-ee844f411152 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:02.062685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993d8d6-8471-4703-85e4-ee844f411152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:02.064498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993d8d6-8471-4703-85e4-ee844f411152 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:02.086409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 131/258] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:02.088979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993d8d6-8471-4703-85e4-ee844f411152 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:02.092146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f77e96-dc29-4f28-b973-2dfeba81c08c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:02.191510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d9982-fcd1-4435-a878-3edd8b151d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:02.192131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9982-fcd1-4435-a878-3edd8b151d4f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:02.192367 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9982-fcd1-4435-a878-3edd8b151d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:02.192606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9982-fcd1-4435-a878-3edd8b151d4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:02.200854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993d8d6-8471-4703-85e4-ee844f411152 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:02.234884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d9982-fcd1-4435-a878-3edd8b151d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:02.237054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa655c3-493e-415d-a861-9c067c7e0b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:02.238524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa655c3-493e-415d-a861-9c067c7e0b4c None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:02.239052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa655c3-493e-415d-a861-9c067c7e0b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:02.239511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa655c3-493e-415d-a861-9c067c7e0b4c None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63/users/1417f9fa5c2c4c2dbc09109785b66b29/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-265d9982-fcd1-4435-a878-3edd8b151d4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:02.242268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:02.243207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:24:02.244005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:02.246260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 128/259] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:02.247455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d9982-fcd1-4435-a878-3edd8b151d4f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:02.256499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa655c3-493e-415d-a861-9c067c7e0b4c None admin] RBAC: Authorizing `identity:create_grant(project_id=ce294031877d49d18dd64a30dbcc7c63, user_id=1417f9fa5c2c4c2dbc09109785b66b29, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:02.256499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa655c3-493e-415d-a861-9c067c7e0b4c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:02.259904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:02.375083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 132/260] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:24:02 2022] PUT /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63/users/1417f9fa5c2c4c2dbc09109785b66b29/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:02.375644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:02.379047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa655c3-493e-415d-a861-9c067c7e0b4c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:02.387403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d9982-fcd1-4435-a878-3edd8b151d4f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:02.478689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f2df8-f848-434c-80fc-ac384f8cc592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:02.479320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f2df8-f848-434c-80fc-ac384f8cc592 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:02.479601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f2df8-f848-434c-80fc-ac384f8cc592 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:02.479871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f2df8-f848-434c-80fc-ac384f8cc592 None admin] PATH_INFO: `/v3/users/d30c81fed7e04452bf3b501a0778eefd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:02.482421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f2df8-f848-434c-80fc-ac384f8cc592 None admin] RBAC: Authorizing `identity:get_user(user_id=d30c81fed7e04452bf3b501a0778eefd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:02.483929 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f2df8-f848-434c-80fc-ac384f8cc592 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:02.491738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 129/261] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:02 2022] GET /identity/v3/users/d30c81fed7e04452bf3b501a0778eefd => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:02.497282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f2df8-f848-434c-80fc-ac384f8cc592 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:02.601605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa655c3-493e-415d-a861-9c067c7e0b4c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:02.641079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7c3c0-230f-4f5a-a696-0dd93d8d9a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:02.641693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7c3c0-230f-4f5a-a696-0dd93d8d9a7a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:02.642024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7c3c0-230f-4f5a-a696-0dd93d8d9a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:02.642326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7c3c0-230f-4f5a-a696-0dd93d8d9a7a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:02.643053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7c3c0-230f-4f5a-a696-0dd93d8d9a7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:02.644034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7c3c0-230f-4f5a-a696-0dd93d8d9a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:02.645565 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7c3c0-230f-4f5a-a696-0dd93d8d9a7a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:02.647593 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:02.656199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 133/262] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:02.662539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7c3c0-230f-4f5a-a696-0dd93d8d9a7a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:02.718767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f2df8-f848-434c-80fc-ac384f8cc592 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:02.775818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af03d7a-dc9c-46f4-b2b4-0542c86c7a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:02.776670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af03d7a-dc9c-46f4-b2b4-0542c86c7a2f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:02.777096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af03d7a-dc9c-46f4-b2b4-0542c86c7a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:02.777517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af03d7a-dc9c-46f4-b2b4-0542c86c7a2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:02.806525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:02.872395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7c3c0-230f-4f5a-a696-0dd93d8d9a7a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:02.913272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af03d7a-dc9c-46f4-b2b4-0542c86c7a2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:02.914967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af03d7a-dc9c-46f4-b2b4-0542c86c7a2f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:02.979431 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8aa8e-e35c-4f97-a46a-fa2af4dd39d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:02.980081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8aa8e-e35c-4f97-a46a-fa2af4dd39d3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:02.980360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8aa8e-e35c-4f97-a46a-fa2af4dd39d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:02.980631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8aa8e-e35c-4f97-a46a-fa2af4dd39d3 None admin] PATH_INFO: `/v3/users/d30c81fed7e04452bf3b501a0778eefd` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:02.982832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8aa8e-e35c-4f97-a46a-fa2af4dd39d3 None admin] RBAC: Authorizing `identity:get_user(user_id=d30c81fed7e04452bf3b501a0778eefd)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:02.984293 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8aa8e-e35c-4f97-a46a-fa2af4dd39d3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:02.990107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 134/263] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:02 2022] GET /identity/v3/users/d30c81fed7e04452bf3b501a0778eefd => generated 283 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:02.991169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8aa8e-e35c-4f97-a46a-fa2af4dd39d3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:03.064682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:03.136709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 130/264] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:24:03.142169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af03d7a-dc9c-46f4-b2b4-0542c86c7a2f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:03.198586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8aa8e-e35c-4f97-a46a-fa2af4dd39d3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:03.250529 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:03.255155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068319ed-5ecf-4715-b325-60ef1d85df11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:03.255830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068319ed-5ecf-4715-b325-60ef1d85df11 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:03.256400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068319ed-5ecf-4715-b325-60ef1d85df11 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:03.256533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068319ed-5ecf-4715-b325-60ef1d85df11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:03.295003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af03d7a-dc9c-46f4-b2b4-0542c86c7a2f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:03.330837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0120b-3c2b-448b-8be1-aed6bc11dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:03.332346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0120b-3c2b-448b-8be1-aed6bc11dc0b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:03.332859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0120b-3c2b-448b-8be1-aed6bc11dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:03.333328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0120b-3c2b-448b-8be1-aed6bc11dc0b None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:03.334684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0120b-3c2b-448b-8be1-aed6bc11dc0b None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:03.336332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0120b-3c2b-448b-8be1-aed6bc11dc0b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:03.338493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 131/265] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:03 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:03.347121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0120b-3c2b-448b-8be1-aed6bc11dc0b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:03.402269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068319ed-5ecf-4715-b325-60ef1d85df11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:03.404368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068319ed-5ecf-4715-b325-60ef1d85df11 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:03.425387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:03.544887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0120b-3c2b-448b-8be1-aed6bc11dc0b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:03.619014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54300af-ac98-40c6-bd07-5ed057fd0c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:03.619648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54300af-ac98-40c6-bd07-5ed057fd0c40 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:03.619921 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54300af-ac98-40c6-bd07-5ed057fd0c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:03.620200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54300af-ac98-40c6-bd07-5ed057fd0c40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:03.620936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54300af-ac98-40c6-bd07-5ed057fd0c40 None admin] RBAC: Adding query filter params (role.id=cae161e9c1b64656924448dd784ed4d1, scope.project.id=ce294031877d49d18dd64a30dbcc7c63, user.id=d30c81fed7e04452bf3b501a0778eefd) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:03.621240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54300af-ac98-40c6-bd07-5ed057fd0c40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:03.622720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54300af-ac98-40c6-bd07-5ed057fd0c40 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:03.630058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 132/266] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:03 2022] GET /identity/v3/role_assignments?user.id=d30c81fed7e04452bf3b501a0778eefd&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=cae161e9c1b64656924448dd784ed4d1 => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:03.639605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e65a7-1041-439c-972c-65b64bcc3c77 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:03.639903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e65a7-1041-439c-972c-65b64bcc3c77 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:03.640170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e65a7-1041-439c-972c-65b64bcc3c77 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:03.641731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 133/267] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:03.642974 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 135/268] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:24:03.648468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606193-12db-4f1a-8746-c54986b4c91c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:03.648732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606193-12db-4f1a-8746-c54986b4c91c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:03.649001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606193-12db-4f1a-8746-c54986b4c91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:03.653701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068319ed-5ecf-4715-b325-60ef1d85df11 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:03.691890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-28606193-12db-4f1a-8746-c54986b4c91c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:03.693119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-28606193-12db-4f1a-8746-c54986b4c91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:03.717728 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:03.823156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068319ed-5ecf-4715-b325-60ef1d85df11 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:03.843388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 134/269] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:03.844551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85990-f647-4c80-b861-e19d61c781d7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:03.844839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85990-f647-4c80-b861-e19d61c781d7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:03.845109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85990-f647-4c80-b861-e19d61c781d7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:03.846775 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 135/270] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:03.852058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5e9f6-2d13-4f96-ac1c-f9d37af3ec78 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:03.852342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5e9f6-2d13-4f96-ac1c-f9d37af3ec78 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:03.852616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5e9f6-2d13-4f96-ac1c-f9d37af3ec78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:03.878488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5d1d9-b0c0-4e8e-bf82-5d3eccbaaf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:03.878488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5d1d9-b0c0-4e8e-bf82-5d3eccbaaf97 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:03.878488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5d1d9-b0c0-4e8e-bf82-5d3eccbaaf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:03.878488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5d1d9-b0c0-4e8e-bf82-5d3eccbaaf97 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:03.878865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9da5e9f6-2d13-4f96-ac1c-f9d37af3ec78 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:03.878865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9da5e9f6-2d13-4f96-ac1c-f9d37af3ec78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:03.878989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5d1d9-b0c0-4e8e-bf82-5d3eccbaaf97 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:03.878989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5d1d9-b0c0-4e8e-bf82-5d3eccbaaf97 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:03.881002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 136/271] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:03 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:03.890505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5d1d9-b0c0-4e8e-bf82-5d3eccbaaf97 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:03.906999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:03.986286 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 136/272] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:04.000834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:04.005923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5d1d9-b0c0-4e8e-bf82-5d3eccbaaf97 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:04.042872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6d18f-965c-460e-9456-2c6c20c73974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:04.043996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6d18f-965c-460e-9456-2c6c20c73974 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:04.044589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6d18f-965c-460e-9456-2c6c20c73974 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:04.045069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6d18f-965c-460e-9456-2c6c20c73974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:04.046002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6d18f-965c-460e-9456-2c6c20c73974 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=ce294031877d49d18dd64a30dbcc7c63, user.id=d30c81fed7e04452bf3b501a0778eefd) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:04.046526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6d18f-965c-460e-9456-2c6c20c73974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:04.048132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6d18f-965c-460e-9456-2c6c20c73974 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:04.054858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 137/273] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:03 2022] GET /identity/v3/role_assignments?user.id=d30c81fed7e04452bf3b501a0778eefd&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:04.175909 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5e9f6-2d13-4f96-ac1c-f9d37af3ec78 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:04.249020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635bdc4-25c7-4636-b343-375509e842c0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:04.249648 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635bdc4-25c7-4636-b343-375509e842c0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:04.250115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635bdc4-25c7-4636-b343-375509e842c0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:04.251917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 138/274] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:04.257885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd983f68-964b-4963-bf19-04ac08de00d2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:04.258440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd983f68-964b-4963-bf19-04ac08de00d2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:04.258904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd983f68-964b-4963-bf19-04ac08de00d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:04.279536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cd983f68-964b-4963-bf19-04ac08de00d2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:04.281153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cd983f68-964b-4963-bf19-04ac08de00d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:04.333808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5e9f6-2d13-4f96-ac1c-f9d37af3ec78 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:04.356184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 139/275] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:04.377486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd983f68-964b-4963-bf19-04ac08de00d2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:04.385042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6501b-bbb3-4a21-944f-0ab4f87781b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:04.390284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6501b-bbb3-4a21-944f-0ab4f87781b8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:04.390284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6501b-bbb3-4a21-944f-0ab4f87781b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:04.390284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6501b-bbb3-4a21-944f-0ab4f87781b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:04.391965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6501b-bbb3-4a21-944f-0ab4f87781b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:04.395443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6501b-bbb3-4a21-944f-0ab4f87781b8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:04.397533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 137/276] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:04.403141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6501b-bbb3-4a21-944f-0ab4f87781b8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:04.461570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:04.529368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:04.540657 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd983f68-964b-4963-bf19-04ac08de00d2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:04.560824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6501b-bbb3-4a21-944f-0ab4f87781b8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:04.581015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be55ac-4154-4f03-b49e-aea8e3018a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:04.581663 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be55ac-4154-4f03-b49e-aea8e3018a47 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:04.581936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be55ac-4154-4f03-b49e-aea8e3018a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:04.582249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be55ac-4154-4f03-b49e-aea8e3018a47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:04.592586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be55ac-4154-4f03-b49e-aea8e3018a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-84be55ac-4154-4f03-b49e-aea8e3018a47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:04.597504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:04.598438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:04.599217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 140/277] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:04.604715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998763b-6d9f-4d1f-bcb9-117ca3d7a14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:04.604715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998763b-6d9f-4d1f-bcb9-117ca3d7a14d None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:04.604715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998763b-6d9f-4d1f-bcb9-117ca3d7a14d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:04.604715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998763b-6d9f-4d1f-bcb9-117ca3d7a14d None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:04.605066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998763b-6d9f-4d1f-bcb9-117ca3d7a14d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:04.605066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998763b-6d9f-4d1f-bcb9-117ca3d7a14d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:04.605184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be55ac-4154-4f03-b49e-aea8e3018a47 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:04.716416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be55ac-4154-4f03-b49e-aea8e3018a47 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:04.741594 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 138/278] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:24:04 2022] POST /identity/v3/users => generated 293 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:24:04.742443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998763b-6d9f-4d1f-bcb9-117ca3d7a14d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:04.764750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f29ddd-e767-467b-b377-215fefadc4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:04.766014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f29ddd-e767-467b-b377-215fefadc4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:04.767076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f29ddd-e767-467b-b377-215fefadc4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:04.767579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f29ddd-e767-467b-b377-215fefadc4b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:04.768937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f29ddd-e767-467b-b377-215fefadc4b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:04.769447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f29ddd-e767-467b-b377-215fefadc4b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:04.771975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f29ddd-e767-467b-b377-215fefadc4b5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:04.785823 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 141/279] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:04.792428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f29ddd-e767-467b-b377-215fefadc4b5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:04.819400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:04.855970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:04.873601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998763b-6d9f-4d1f-bcb9-117ca3d7a14d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:04.927175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f188c-9d3d-469f-a40b-9457fc98842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:04.928500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f188c-9d3d-469f-a40b-9457fc98842c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:04.929503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f188c-9d3d-469f-a40b-9457fc98842c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:04.930114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f188c-9d3d-469f-a40b-9457fc98842c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:04.938922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f29ddd-e767-467b-b377-215fefadc4b5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:04.950893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f188c-9d3d-469f-a40b-9457fc98842c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-676f188c-9d3d-469f-a40b-9457fc98842c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:04.958274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:04.959272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:24:04.960104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:04.962384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 139/280] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:04.968378 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f188c-9d3d-469f-a40b-9457fc98842c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:04.988616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a9ec7-d926-41be-a6e9-d2ef2f362f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:04.989374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a9ec7-d926-41be-a6e9-d2ef2f362f1c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:04.991862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a9ec7-d926-41be-a6e9-d2ef2f362f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:04.991862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a9ec7-d926-41be-a6e9-d2ef2f362f1c None admin] PATH_INFO: `/v3/users/cae82ab763d546708d0c263651802760` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:05.012366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a9ec7-d926-41be-a6e9-d2ef2f362f1c None admin] RBAC: Authorizing `identity:get_user(user_id=cae82ab763d546708d0c263651802760)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:05.013835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a9ec7-d926-41be-a6e9-d2ef2f362f1c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:05.015868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 142/281] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:04 2022] GET /identity/v3/users/cae82ab763d546708d0c263651802760 => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:05.021563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a9ec7-d926-41be-a6e9-d2ef2f362f1c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:05.108207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f188c-9d3d-469f-a40b-9457fc98842c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:05.110249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:05.154841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a9ec7-d926-41be-a6e9-d2ef2f362f1c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:05.165044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2b492-a2b8-4ef6-8ece-2c2d572bcde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:05.165662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2b492-a2b8-4ef6-8ece-2c2d572bcde5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:05.167092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2b492-a2b8-4ef6-8ece-2c2d572bcde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:05.167092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2b492-a2b8-4ef6-8ece-2c2d572bcde5 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:05.167092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2b492-a2b8-4ef6-8ece-2c2d572bcde5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:05.167284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2b492-a2b8-4ef6-8ece-2c2d572bcde5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:05.168510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2b492-a2b8-4ef6-8ece-2c2d572bcde5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:05.181969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 140/282] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:05.189172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2b492-a2b8-4ef6-8ece-2c2d572bcde5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:05.203654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:05.219088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a191f74-44d1-46cc-8691-25fdd4c440fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:05.220129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a191f74-44d1-46cc-8691-25fdd4c440fd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:05.220619 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a191f74-44d1-46cc-8691-25fdd4c440fd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:05.221086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a191f74-44d1-46cc-8691-25fdd4c440fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:05.222764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a191f74-44d1-46cc-8691-25fdd4c440fd None admin] RBAC: Adding query filter params (role.id=1e04fdc0cc334fd7a733a1490a194e37, user.id=cae82ab763d546708d0c263651802760, scope.system=all) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:05.223289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a191f74-44d1-46cc-8691-25fdd4c440fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:05.225680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a191f74-44d1-46cc-8691-25fdd4c440fd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:05.232308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 143/283] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Oct 4 05:24:05 2022] GET /identity/v3/role_assignments?user.id=cae82ab763d546708d0c263651802760&scope.system=all&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:05.314430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2b492-a2b8-4ef6-8ece-2c2d572bcde5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:05.325402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed24d9b-1d51-4d67-beab-f828d44490dc None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:05.326454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed24d9b-1d51-4d67-beab-f828d44490dc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:05.326951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed24d9b-1d51-4d67-beab-f828d44490dc None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:05.329920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 144/284] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:05.336504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4b692-6df2-4894-9d2a-f386015d5989 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:05.337092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4b692-6df2-4894-9d2a-f386015d5989 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:05.337558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4b692-6df2-4894-9d2a-f386015d5989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:05.361871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0df4b692-6df2-4894-9d2a-f386015d5989 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:05.363710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0df4b692-6df2-4894-9d2a-f386015d5989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:05.364724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b78a9e-260a-4910-8577-b104cde60a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:05.365380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b78a9e-260a-4910-8577-b104cde60a89 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:05.365654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b78a9e-260a-4910-8577-b104cde60a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:05.365923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b78a9e-260a-4910-8577-b104cde60a89 None admin] PATH_INFO: `/v3/users/9b9d59b9940e419c9086fd47cfcf07ee` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:05.367215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b78a9e-260a-4910-8577-b104cde60a89 None admin] RBAC: Authorizing `identity:get_user(user_id=9b9d59b9940e419c9086fd47cfcf07ee)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:05.370109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b78a9e-260a-4910-8577-b104cde60a89 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:05.372122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 141/285] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:05 2022] GET /identity/v3/users/9b9d59b9940e419c9086fd47cfcf07ee => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:05.381220 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b78a9e-260a-4910-8577-b104cde60a89 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:05.477794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 145/286] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:05.505064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:05.529926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b78a9e-260a-4910-8577-b104cde60a89 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:05.566314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eedd16-8914-49d7-ba37-df9c27b7f25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:05.567311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eedd16-8914-49d7-ba37-df9c27b7f25d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:05.567789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eedd16-8914-49d7-ba37-df9c27b7f25d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:05.568541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eedd16-8914-49d7-ba37-df9c27b7f25d None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:05.569862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eedd16-8914-49d7-ba37-df9c27b7f25d None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:05.571521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eedd16-8914-49d7-ba37-df9c27b7f25d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:05.573659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 142/287] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:05 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:05.579689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4b692-6df2-4894-9d2a-f386015d5989 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:05.647853 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:05.695932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4b692-6df2-4894-9d2a-f386015d5989 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:05.740757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5eff9-ec4d-411b-ab70-f65cff386844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:05.741747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5eff9-ec4d-411b-ab70-f65cff386844 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:05.742263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5eff9-ec4d-411b-ab70-f65cff386844 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:05.742747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5eff9-ec4d-411b-ab70-f65cff386844 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63/users/9b9d59b9940e419c9086fd47cfcf07ee/roles/840afb6280c341c6beefc211b040c463` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:05.744837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5eff9-ec4d-411b-ab70-f65cff386844 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce294031877d49d18dd64a30dbcc7c63, user_id=9b9d59b9940e419c9086fd47cfcf07ee, role_id=840afb6280c341c6beefc211b040c463)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:05.746480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5eff9-ec4d-411b-ab70-f65cff386844 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:05.840556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eedd16-8914-49d7-ba37-df9c27b7f25d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:05.843751 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 146/288] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:24:05 2022] PUT /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63/users/9b9d59b9940e419c9086fd47cfcf07ee/roles/840afb6280c341c6beefc211b040c463 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:06.068924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eedd16-8914-49d7-ba37-df9c27b7f25d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:06.108040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41672a19-9e6b-4328-9048-648e254ebef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:06.108679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41672a19-9e6b-4328-9048-648e254ebef2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:06.108961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41672a19-9e6b-4328-9048-648e254ebef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:06.109237 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41672a19-9e6b-4328-9048-648e254ebef2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:06.122470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41672a19-9e6b-4328-9048-648e254ebef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-41672a19-9e6b-4328-9048-648e254ebef2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:06.129016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:06.129971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:06.130786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 143/289] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:06.136452 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5eff9-ec4d-411b-ab70-f65cff386844 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:06.242693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5eff9-ec4d-411b-ab70-f65cff386844 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:06.274810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:06.286699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75c146-76ad-48e6-8a3b-e95ccfa4f8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:06.287359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75c146-76ad-48e6-8a3b-e95ccfa4f8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:06.293557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75c146-76ad-48e6-8a3b-e95ccfa4f8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:06.293557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75c146-76ad-48e6-8a3b-e95ccfa4f8e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:06.293557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75c146-76ad-48e6-8a3b-e95ccfa4f8e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:06.293557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75c146-76ad-48e6-8a3b-e95ccfa4f8e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:06.293557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75c146-76ad-48e6-8a3b-e95ccfa4f8e8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:06.302723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 147/290] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:06.308269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41672a19-9e6b-4328-9048-648e254ebef2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:06.312217 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:06.439511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41672a19-9e6b-4328-9048-648e254ebef2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:06.489184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6c25-dd1a-47d6-99cd-0feb48a44ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:06.490179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6c25-dd1a-47d6-99cd-0feb48a44ffb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:06.490683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6c25-dd1a-47d6-99cd-0feb48a44ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:06.491142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6c25-dd1a-47d6-99cd-0feb48a44ffb None admin] PATH_INFO: `/v3/users/1417f9fa5c2c4c2dbc09109785b66b29` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:06.492467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb6c25-dd1a-47d6-99cd-0feb48a44ffb None admin] RBAC: Authorizing `identity:get_user(user_id=1417f9fa5c2c4c2dbc09109785b66b29)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:06.494101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb6c25-dd1a-47d6-99cd-0feb48a44ffb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:06.495944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 144/291] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:06 2022] GET /identity/v3/users/1417f9fa5c2c4c2dbc09109785b66b29 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:06.501424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75c146-76ad-48e6-8a3b-e95ccfa4f8e8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:06.618101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75c146-76ad-48e6-8a3b-e95ccfa4f8e8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:06.658200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ba6ce-9082-482f-bc6e-1d07cf175f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:06.658811 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ba6ce-9082-482f-bc6e-1d07cf175f35 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:06.659641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ba6ce-9082-482f-bc6e-1d07cf175f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:06.659917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ba6ce-9082-482f-bc6e-1d07cf175f35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:06.716128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:06.732584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ba6ce-9082-482f-bc6e-1d07cf175f35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:06.734954 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ba6ce-9082-482f-bc6e-1d07cf175f35 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:06.860159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 148/292] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:06.865758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6c25-dd1a-47d6-99cd-0feb48a44ffb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:06.937264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:06.968247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6c25-dd1a-47d6-99cd-0feb48a44ffb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:07.004227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a6715-4aa1-4693-b0c8-068c02a75535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:07.004884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a6715-4aa1-4693-b0c8-068c02a75535 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:07.005157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a6715-4aa1-4693-b0c8-068c02a75535 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:07.005420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a6715-4aa1-4693-b0c8-068c02a75535 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:07.006606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a6715-4aa1-4693-b0c8-068c02a75535 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:07.008033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a6715-4aa1-4693-b0c8-068c02a75535 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:07.010065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 145/293] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:06 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:07.015909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ba6ce-9082-482f-bc6e-1d07cf175f35 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:07.106203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:07.119661 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ba6ce-9082-482f-bc6e-1d07cf175f35 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:07.154458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10bb2d-118c-4239-854d-aa4458f25032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:07.155065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10bb2d-118c-4239-854d-aa4458f25032 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:07.155459 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10bb2d-118c-4239-854d-aa4458f25032 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:07.155736 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10bb2d-118c-4239-854d-aa4458f25032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:07.156485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10bb2d-118c-4239-854d-aa4458f25032 None admin] RBAC: Adding query filter params (role.id=1e04fdc0cc334fd7a733a1490a194e37, scope.project.id=ce294031877d49d18dd64a30dbcc7c63, user.id=1417f9fa5c2c4c2dbc09109785b66b29) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:07.156792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10bb2d-118c-4239-854d-aa4458f25032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:07.158244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10bb2d-118c-4239-854d-aa4458f25032 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:07.164726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 149/294] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:07 2022] GET /identity/v3/role_assignments?user.id=1417f9fa5c2c4c2dbc09109785b66b29&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:07.692812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcc74e-4883-4636-802c-985f6e897505 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:07.693572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcc74e-4883-4636-802c-985f6e897505 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:07.693857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fcc74e-4883-4636-802c-985f6e897505 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:07.695168 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 146/295] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:07.702609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054a160-f874-47b3-8747-eeda493c0ccd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:07.703030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054a160-f874-47b3-8747-eeda493c0ccd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:07.703202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054a160-f874-47b3-8747-eeda493c0ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:07.721674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ab3d0-d106-49f1-9ca7-06628bd97b87 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:07.721953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ab3d0-d106-49f1-9ca7-06628bd97b87 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:07.722072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9054a160-f874-47b3-8747-eeda493c0ccd None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:07.722253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ab3d0-d106-49f1-9ca7-06628bd97b87 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:07.723348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9054a160-f874-47b3-8747-eeda493c0ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:07.723871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 147/296] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:07.729440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c787bbf-2198-4c1b-a6e2-f92bf77a53a6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:07.729725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c787bbf-2198-4c1b-a6e2-f92bf77a53a6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:07.730003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c787bbf-2198-4c1b-a6e2-f92bf77a53a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:07.749068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4c787bbf-2198-4c1b-a6e2-f92bf77a53a6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:07.750341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4c787bbf-2198-4c1b-a6e2-f92bf77a53a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:07.809826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 150/297] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:07.839032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 148/298] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:07.880233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023e1a-8d50-4d7f-8bf0-8c5eaa3f5c29 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:07.880529 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023e1a-8d50-4d7f-8bf0-8c5eaa3f5c29 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:07.880794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023e1a-8d50-4d7f-8bf0-8c5eaa3f5c29 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:07.882260 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 151/299] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:07.893213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34fb1c-5990-4cb3-80dd-9e0d43b0c099 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:07.893213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34fb1c-5990-4cb3-80dd-9e0d43b0c099 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:07.893213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34fb1c-5990-4cb3-80dd-9e0d43b0c099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:07.910873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9c34fb1c-5990-4cb3-80dd-9e0d43b0c099 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:07.912284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9c34fb1c-5990-4cb3-80dd-9e0d43b0c099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:07.977246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:07.981778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 149/300] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:08.101107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd023e1a-8d50-4d7f-8bf0-8c5eaa3f5c29 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:08.118646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34fb1c-5990-4cb3-80dd-9e0d43b0c099 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:08.194292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:08.217384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd023e1a-8d50-4d7f-8bf0-8c5eaa3f5c29 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:08.229837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34fb1c-5990-4cb3-80dd-9e0d43b0c099 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:08.258797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7f078-0210-45e4-8609-723730a05b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:08.259324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7f078-0210-45e4-8609-723730a05b27 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:08.259598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7f078-0210-45e4-8609-723730a05b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:08.259867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7f078-0210-45e4-8609-723730a05b27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:08.270597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7f078-0210-45e4-8609-723730a05b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:08.270915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4300e-0800-4f61-8919-9689ab2df525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:08.271550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4300e-0800-4f61-8919-9689ab2df525 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:08.271828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4300e-0800-4f61-8919-9689ab2df525 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:08.272098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4300e-0800-4f61-8919-9689ab2df525 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c4d7f078-0210-45e4-8609-723730a05b27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:08.275376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:08.276270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:24:08.277076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:08.281250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 152/301] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:08.282082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7f078-0210-45e4-8609-723730a05b27 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:08.284942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4300e-0800-4f61-8919-9689ab2df525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d1b4300e-0800-4f61-8919-9689ab2df525 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:08.289842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:08.290754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:08.291557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:08.295673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 150/302] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:08.296413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4300e-0800-4f61-8919-9689ab2df525 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:08.358339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:08.392461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7f078-0210-45e4-8609-723730a05b27 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:08.404193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4300e-0800-4f61-8919-9689ab2df525 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:08.433271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98b1e5-ea6c-401f-aed0-67ca9d348e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:08.433902 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98b1e5-ea6c-401f-aed0-67ca9d348e3b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:08.434206 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98b1e5-ea6c-401f-aed0-67ca9d348e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:08.434477 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98b1e5-ea6c-401f-aed0-67ca9d348e3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:08.435712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98b1e5-ea6c-401f-aed0-67ca9d348e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:08.437138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98b1e5-ea6c-401f-aed0-67ca9d348e3b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:08.438729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 153/303] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:08.439486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98b1e5-ea6c-401f-aed0-67ca9d348e3b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:08.445802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec92aa7-5ccf-4da0-9fc1-ec167e1b9604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:08.446435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec92aa7-5ccf-4da0-9fc1-ec167e1b9604 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:08.446734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec92aa7-5ccf-4da0-9fc1-ec167e1b9604 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:08.447817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec92aa7-5ccf-4da0-9fc1-ec167e1b9604 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:08.448488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec92aa7-5ccf-4da0-9fc1-ec167e1b9604 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:08.448792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec92aa7-5ccf-4da0-9fc1-ec167e1b9604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:08.450180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec92aa7-5ccf-4da0-9fc1-ec167e1b9604 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:08.458375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:08.459465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 151/304] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:08.460227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec92aa7-5ccf-4da0-9fc1-ec167e1b9604 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:08.554122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98b1e5-ea6c-401f-aed0-67ca9d348e3b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:08.562433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec92aa7-5ccf-4da0-9fc1-ec167e1b9604 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:08.584691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:08.587656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca400554-eaa8-46b6-abba-efe7c9fb14ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:08.588260 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca400554-eaa8-46b6-abba-efe7c9fb14ae None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:08.588544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca400554-eaa8-46b6-abba-efe7c9fb14ae None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:08.588812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca400554-eaa8-46b6-abba-efe7c9fb14ae None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:08.589462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca400554-eaa8-46b6-abba-efe7c9fb14ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:08.589762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca400554-eaa8-46b6-abba-efe7c9fb14ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:08.591180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca400554-eaa8-46b6-abba-efe7c9fb14ae None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:08.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d4cee-036d-4726-a9fc-4b451ce167ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:08.596351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d4cee-036d-4726-a9fc-4b451ce167ce None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:08.596636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d4cee-036d-4726-a9fc-4b451ce167ce None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:08.596906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d4cee-036d-4726-a9fc-4b451ce167ce None admin] PATH_INFO: `/v3/groups` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:08.597530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d4cee-036d-4726-a9fc-4b451ce167ce None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:08.598973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d4cee-036d-4726-a9fc-4b451ce167ce None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:08.600202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 154/305] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:08.600983 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca400554-eaa8-46b6-abba-efe7c9fb14ae None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:08.670497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:08.698771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca400554-eaa8-46b6-abba-efe7c9fb14ae None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:08.706808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 152/306] 10.222.0.26 () {60 vars in 1134 bytes} [Tue Oct 4 05:24:08 2022] POST /identity/v3/groups => generated 229 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:24:08.708051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc0487-1b98-4e42-ac7e-42ba9ba73ef0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:08.708336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc0487-1b98-4e42-ac7e-42ba9ba73ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:08.708612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc0487-1b98-4e42-ac7e-42ba9ba73ef0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:08.710050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 153/307] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:08.710619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc0487-1b98-4e42-ac7e-42ba9ba73ef0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:08.740823 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153254f1-6922-4aa9-b8cb-09256f6be683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:08.741458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153254f1-6922-4aa9-b8cb-09256f6be683 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:08.741744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153254f1-6922-4aa9-b8cb-09256f6be683 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:08.742016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153254f1-6922-4aa9-b8cb-09256f6be683 None admin] PATH_INFO: `/v3/users/64c1e42e23694714a5c89ee380e98da5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:08.757983 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153254f1-6922-4aa9-b8cb-09256f6be683 None admin] RBAC: Authorizing `identity:get_user(user_id=64c1e42e23694714a5c89ee380e98da5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:08.759694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153254f1-6922-4aa9-b8cb-09256f6be683 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:08.766811 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 155/308] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:08 2022] GET /identity/v3/users/64c1e42e23694714a5c89ee380e98da5 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:08.766811 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc8078-4961-4f94-9c62-61e70b11839a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:08.766811 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc8078-4961-4f94-9c62-61e70b11839a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:08.766811 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc8078-4961-4f94-9c62-61e70b11839a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:08.783780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e0bc8078-4961-4f94-9c62-61e70b11839a None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:08.785074 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e0bc8078-4961-4f94-9c62-61e70b11839a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:08.849780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc0487-1b98-4e42-ac7e-42ba9ba73ef0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:08.866634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 156/309] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:08.874997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc8078-4961-4f94-9c62-61e70b11839a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:08.898291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923b43c-a75e-4844-8cb5-8f4d33ced1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:08.899352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b43c-a75e-4844-8cb5-8f4d33ced1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:08.899640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b43c-a75e-4844-8cb5-8f4d33ced1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:08.900855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b43c-a75e-4844-8cb5-8f4d33ced1e7 None admin] PATH_INFO: `/v3/users/cae82ab763d546708d0c263651802760` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:08.901539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923b43c-a75e-4844-8cb5-8f4d33ced1e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cae82ab763d546708d0c263651802760)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:08.903546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923b43c-a75e-4844-8cb5-8f4d33ced1e7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:08.905675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 154/310] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:08 2022] GET /identity/v3/users/cae82ab763d546708d0c263651802760 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:08.911789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923b43c-a75e-4844-8cb5-8f4d33ced1e7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:08.947988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:08.985482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:08.995194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc8078-4961-4f94-9c62-61e70b11839a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:09.030464 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b204b-8ea8-4b19-8e1a-341ba4b65b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:09.031251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b204b-8ea8-4b19-8e1a-341ba4b65b8e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:09.031597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b204b-8ea8-4b19-8e1a-341ba4b65b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:09.031935 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b204b-8ea8-4b19-8e1a-341ba4b65b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:09.032778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b204b-8ea8-4b19-8e1a-341ba4b65b8e None admin] RBAC: Adding query filter params (role.id=840afb6280c341c6beefc211b040c463, user.id=64c1e42e23694714a5c89ee380e98da5, scope.system=all) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:09.033158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b204b-8ea8-4b19-8e1a-341ba4b65b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:09.034760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b204b-8ea8-4b19-8e1a-341ba4b65b8e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:09.047569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 157/311] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Oct 4 05:24:08 2022] GET /identity/v3/role_assignments?user.id=64c1e42e23694714a5c89ee380e98da5&scope.system=all&role.id=840afb6280c341c6beefc211b040c463 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:09.052385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923b43c-a75e-4844-8cb5-8f4d33ced1e7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:09.093055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec6f25-72f9-4459-9230-372b0671fe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:09.094077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec6f25-72f9-4459-9230-372b0671fe4a None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:09.094518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec6f25-72f9-4459-9230-372b0671fe4a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:09.094936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec6f25-72f9-4459-9230-372b0671fe4a None admin] PATH_INFO: `/v3/system/users/cae82ab763d546708d0c263651802760/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:09.096766 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec6f25-72f9-4459-9230-372b0671fe4a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cae82ab763d546708d0c263651802760, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:09.098438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec6f25-72f9-4459-9230-372b0671fe4a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:09.145705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b204b-8ea8-4b19-8e1a-341ba4b65b8e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:09.176707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 155/312] 10.222.0.26 () {58 vars in 1334 bytes} [Tue Oct 4 05:24:08 2022] PUT /identity/v3/system/users/cae82ab763d546708d0c263651802760/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:24:09.271361 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b204b-8ea8-4b19-8e1a-341ba4b65b8e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:09.298030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:09.309988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbda0c0-301f-4885-b548-ec2b04ed8a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:09.310692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbda0c0-301f-4885-b548-ec2b04ed8a02 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:09.310974 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbda0c0-301f-4885-b548-ec2b04ed8a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:09.311244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbda0c0-301f-4885-b548-ec2b04ed8a02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:09.324160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbda0c0-301f-4885-b548-ec2b04ed8a02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9cbda0c0-301f-4885-b548-ec2b04ed8a02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:09.329296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:09.330262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:09.331085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 158/313] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:09.342435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec6f25-72f9-4459-9230-372b0671fe4a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:09.447609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec6f25-72f9-4459-9230-372b0671fe4a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:09.462684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:09.482408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fcd2e-49a0-4307-a6d2-0abdf8a7f301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:09.483114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fcd2e-49a0-4307-a6d2-0abdf8a7f301 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:09.483424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fcd2e-49a0-4307-a6d2-0abdf8a7f301 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:09.483697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fcd2e-49a0-4307-a6d2-0abdf8a7f301 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:09.484415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559fcd2e-49a0-4307-a6d2-0abdf8a7f301 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:09.484717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559fcd2e-49a0-4307-a6d2-0abdf8a7f301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:09.486091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559fcd2e-49a0-4307-a6d2-0abdf8a7f301 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:09.495896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 156/314] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:09.501618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbda0c0-301f-4885-b548-ec2b04ed8a02 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:09.609714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbda0c0-301f-4885-b548-ec2b04ed8a02 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:09.646409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22193d3b-11f0-44ea-ba1c-63a8b37b8602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:09.647047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22193d3b-11f0-44ea-ba1c-63a8b37b8602 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:09.647329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22193d3b-11f0-44ea-ba1c-63a8b37b8602 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:09.647601 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22193d3b-11f0-44ea-ba1c-63a8b37b8602 None admin] PATH_INFO: `/v3/users/9b9d59b9940e419c9086fd47cfcf07ee` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:09.648734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22193d3b-11f0-44ea-ba1c-63a8b37b8602 None admin] RBAC: Authorizing `identity:get_user(user_id=9b9d59b9940e419c9086fd47cfcf07ee)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:09.650956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22193d3b-11f0-44ea-ba1c-63a8b37b8602 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:09.653400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 159/315] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:09 2022] GET /identity/v3/users/9b9d59b9940e419c9086fd47cfcf07ee => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:09.658765 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fcd2e-49a0-4307-a6d2-0abdf8a7f301 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:09.768046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fcd2e-49a0-4307-a6d2-0abdf8a7f301 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:09.804419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b7225-3efe-4141-8e0a-a2caf277ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:09.805418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b7225-3efe-4141-8e0a-a2caf277ab48 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:09.805892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b7225-3efe-4141-8e0a-a2caf277ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:09.806415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b7225-3efe-4141-8e0a-a2caf277ab48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:09.853999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:09.877792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b7225-3efe-4141-8e0a-a2caf277ab48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:09.879605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b7225-3efe-4141-8e0a-a2caf277ab48 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:09.988132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 157/316] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:09.993854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22193d3b-11f0-44ea-ba1c-63a8b37b8602 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:10.047625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:10.099758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22193d3b-11f0-44ea-ba1c-63a8b37b8602 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:10.138071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d67de-1d35-4a63-8bc3-4f8f891beff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:10.139059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d67de-1d35-4a63-8bc3-4f8f891beff5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:10.139536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d67de-1d35-4a63-8bc3-4f8f891beff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:10.140003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d67de-1d35-4a63-8bc3-4f8f891beff5 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:10.141304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d67de-1d35-4a63-8bc3-4f8f891beff5 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:10.142993 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d67de-1d35-4a63-8bc3-4f8f891beff5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:10.145442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 160/317] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:09 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:10.151373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b7225-3efe-4141-8e0a-a2caf277ab48 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:10.229541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:10.253811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b7225-3efe-4141-8e0a-a2caf277ab48 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:10.288023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad04f1-363e-4968-ace6-a2a0bb86e830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:10.288629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad04f1-363e-4968-ace6-a2a0bb86e830 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:10.288955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad04f1-363e-4968-ace6-a2a0bb86e830 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:10.289225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad04f1-363e-4968-ace6-a2a0bb86e830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:10.289990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad04f1-363e-4968-ace6-a2a0bb86e830 None admin] RBAC: Adding query filter params (role.id=840afb6280c341c6beefc211b040c463, scope.project.id=ce294031877d49d18dd64a30dbcc7c63, user.id=9b9d59b9940e419c9086fd47cfcf07ee) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:10.290320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad04f1-363e-4968-ace6-a2a0bb86e830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:10.291738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad04f1-363e-4968-ace6-a2a0bb86e830 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:10.297859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 158/318] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:10 2022] GET /identity/v3/role_assignments?user.id=9b9d59b9940e419c9086fd47cfcf07ee&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=840afb6280c341c6beefc211b040c463 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:11.027796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79d0e-dd44-4d8c-881a-4c2de514eef0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.028352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79d0e-dd44-4d8c-881a-4c2de514eef0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.028352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79d0e-dd44-4d8c-881a-4c2de514eef0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.029822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 161/319] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:11.034891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93011-ad63-40e4-bd90-bedf71df1607 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.035287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93011-ad63-40e4-bd90-bedf71df1607 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.035443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93011-ad63-40e4-bd90-bedf71df1607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.053566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-57f93011-ad63-40e4-bd90-bedf71df1607 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:11.054849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-57f93011-ad63-40e4-bd90-bedf71df1607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:11.122890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 159/320] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:11.287172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3506a-d2df-46ed-9d05-177c40cacada None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.287518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3506a-d2df-46ed-9d05-177c40cacada None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.287687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3506a-d2df-46ed-9d05-177c40cacada None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.289092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 162/321] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:11.293964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfeb58-04b8-4764-9b81-29b29ebac466 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.294283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfeb58-04b8-4764-9b81-29b29ebac466 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.294561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfeb58-04b8-4764-9b81-29b29ebac466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.312824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-15cfeb58-04b8-4764-9b81-29b29ebac466 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:11.314004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-15cfeb58-04b8-4764-9b81-29b29ebac466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:11.358764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3990f2-e993-4d93-88ac-29034b51b09a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.359063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3990f2-e993-4d93-88ac-29034b51b09a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.359314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3990f2-e993-4d93-88ac-29034b51b09a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.360638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 163/322] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:24:11.366220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4406c8-eba7-4fe7-aad4-5c774f1bd3b0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.366405 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4406c8-eba7-4fe7-aad4-5c774f1bd3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.366682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4406c8-eba7-4fe7-aad4-5c774f1bd3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.380252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:11.384619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 160/323] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:11.387400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-aa4406c8-eba7-4fe7-aad4-5c774f1bd3b0 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:11.388664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aa4406c8-eba7-4fe7-aad4-5c774f1bd3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:11.391487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfeb58-04b8-4764-9b81-29b29ebac466 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:11.467244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 164/324] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:11.505150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfeb58-04b8-4764-9b81-29b29ebac466 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:11.539377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf34b6-6387-49f9-a7fd-f694f2db749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:11.539937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf34b6-6387-49f9-a7fd-f694f2db749b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.540212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf34b6-6387-49f9-a7fd-f694f2db749b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.540482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf34b6-6387-49f9-a7fd-f694f2db749b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.541577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf34b6-6387-49f9-a7fd-f694f2db749b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:11.543042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf34b6-6387-49f9-a7fd-f694f2db749b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:11.544740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 161/325] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:11.550191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4406c8-eba7-4fe7-aad4-5c774f1bd3b0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:11.599599 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:11.648269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf34b6-6387-49f9-a7fd-f694f2db749b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:11.649200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4406c8-eba7-4fe7-aad4-5c774f1bd3b0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:11.684834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac29082-1f36-4a40-af89-1f48b750d7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:11.685446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac29082-1f36-4a40-af89-1f48b750d7aa None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.685721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac29082-1f36-4a40-af89-1f48b750d7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.685991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac29082-1f36-4a40-af89-1f48b750d7aa None admin] PATH_INFO: `/v3/groups` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.686649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac29082-1f36-4a40-af89-1f48b750d7aa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:11.688071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac29082-1f36-4a40-af89-1f48b750d7aa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:11.717702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:11.750056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf34b6-6387-49f9-a7fd-f694f2db749b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:11.777194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 165/326] 10.222.0.26 () {60 vars in 1134 bytes} [Tue Oct 4 05:24:11 2022] POST /identity/v3/groups => generated 226 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:24:11.778022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac29082-1f36-4a40-af89-1f48b750d7aa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:11.791166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bce554-e481-43a6-9100-4febcb6faf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:11.791731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bce554-e481-43a6-9100-4febcb6faf34 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.792010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bce554-e481-43a6-9100-4febcb6faf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.792284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bce554-e481-43a6-9100-4febcb6faf34 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.804610 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bce554-e481-43a6-9100-4febcb6faf34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-39bce554-e481-43a6-9100-4febcb6faf34 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:11.810919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:11.812079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:24:11.813098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:11.815783 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 162/327] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:11.822260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bce554-e481-43a6-9100-4febcb6faf34 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:11.900894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac29082-1f36-4a40-af89-1f48b750d7aa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:11.913008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:11.941025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf389b-8765-485b-9c0a-3205e094221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:11.941681 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf389b-8765-485b-9c0a-3205e094221b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.941962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf389b-8765-485b-9c0a-3205e094221b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.942268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf389b-8765-485b-9c0a-3205e094221b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.943581 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bce554-e481-43a6-9100-4febcb6faf34 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:11.956748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf389b-8765-485b-9c0a-3205e094221b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8baf389b-8765-485b-9c0a-3205e094221b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:11.961769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:11.973910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 166/328] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:11.974910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf389b-8765-485b-9c0a-3205e094221b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:11.976730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:11.985233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1390179-efb5-4f18-a4d2-d754a65099f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:11.985848 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1390179-efb5-4f18-a4d2-d754a65099f0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:11.986158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1390179-efb5-4f18-a4d2-d754a65099f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:11.986428 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1390179-efb5-4f18-a4d2-d754a65099f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:11.987621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1390179-efb5-4f18-a4d2-d754a65099f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:11.988693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1390179-efb5-4f18-a4d2-d754a65099f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:11.993356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1390179-efb5-4f18-a4d2-d754a65099f0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:12.005540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 163/329] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:12.011165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1390179-efb5-4f18-a4d2-d754a65099f0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:12.093548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf389b-8765-485b-9c0a-3205e094221b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:12.134016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1390179-efb5-4f18-a4d2-d754a65099f0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:12.147607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd45802-7ec8-46fb-ba5c-6b097bad686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:12.148223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd45802-7ec8-46fb-ba5c-6b097bad686e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:12.148753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd45802-7ec8-46fb-ba5c-6b097bad686e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:12.149032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd45802-7ec8-46fb-ba5c-6b097bad686e None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:12.150245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd45802-7ec8-46fb-ba5c-6b097bad686e None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:12.150822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd45802-7ec8-46fb-ba5c-6b097bad686e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:12.152497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd45802-7ec8-46fb-ba5c-6b097bad686e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:12.163235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 167/330] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:12.170446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd45802-7ec8-46fb-ba5c-6b097bad686e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:12.179513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980038d-6ba4-4fe3-b17d-c4e974d9cc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:12.180541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980038d-6ba4-4fe3-b17d-c4e974d9cc54 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:12.181446 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980038d-6ba4-4fe3-b17d-c4e974d9cc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:12.181936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980038d-6ba4-4fe3-b17d-c4e974d9cc54 None admin] PATH_INFO: `/v3/users/64c1e42e23694714a5c89ee380e98da5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:12.183372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980038d-6ba4-4fe3-b17d-c4e974d9cc54 None admin] RBAC: Authorizing `identity:get_user(user_id=64c1e42e23694714a5c89ee380e98da5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:12.185616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980038d-6ba4-4fe3-b17d-c4e974d9cc54 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:12.189383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 164/331] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:12 2022] GET /identity/v3/users/64c1e42e23694714a5c89ee380e98da5 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:12.193122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980038d-6ba4-4fe3-b17d-c4e974d9cc54 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:12.212018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:12.273732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:12.313462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd45802-7ec8-46fb-ba5c-6b097bad686e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:12.327214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980038d-6ba4-4fe3-b17d-c4e974d9cc54 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:12.359503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd865c9-2cfe-46ee-9978-737ebf8bb029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:12.360150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd865c9-2cfe-46ee-9978-737ebf8bb029 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:12.360658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd865c9-2cfe-46ee-9978-737ebf8bb029 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:12.360831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd865c9-2cfe-46ee-9978-737ebf8bb029 None admin] PATH_INFO: `/v3/users/cae82ab763d546708d0c263651802760` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:12.362596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd865c9-2cfe-46ee-9978-737ebf8bb029 None admin] RBAC: Authorizing `identity:get_user(user_id=cae82ab763d546708d0c263651802760)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:12.364060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd865c9-2cfe-46ee-9978-737ebf8bb029 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:12.366214 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 168/332] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:12 2022] GET /identity/v3/users/cae82ab763d546708d0c263651802760 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:12.375839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd865c9-2cfe-46ee-9978-737ebf8bb029 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:12.379063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f690c8-14ed-45ef-b464-6a82d88da524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:12.380834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f690c8-14ed-45ef-b464-6a82d88da524 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:12.381283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f690c8-14ed-45ef-b464-6a82d88da524 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:12.381698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f690c8-14ed-45ef-b464-6a82d88da524 None admin] PATH_INFO: `/v3/system/users/64c1e42e23694714a5c89ee380e98da5/roles/840afb6280c341c6beefc211b040c463` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:12.384173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f690c8-14ed-45ef-b464-6a82d88da524 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=64c1e42e23694714a5c89ee380e98da5, role_id=840afb6280c341c6beefc211b040c463)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:12.386277 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f690c8-14ed-45ef-b464-6a82d88da524 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:12.470696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 165/333] 10.222.0.26 () {58 vars in 1334 bytes} [Tue Oct 4 05:24:12 2022] PUT /identity/v3/system/users/64c1e42e23694714a5c89ee380e98da5/roles/840afb6280c341c6beefc211b040c463 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:12.494368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:12.498436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd865c9-2cfe-46ee-9978-737ebf8bb029 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:12.540562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a991413-4a9c-449a-99cd-400282d3542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:12.541180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a991413-4a9c-449a-99cd-400282d3542b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:12.541463 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a991413-4a9c-449a-99cd-400282d3542b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:12.541741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a991413-4a9c-449a-99cd-400282d3542b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:12.550165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a991413-4a9c-449a-99cd-400282d3542b None admin] RBAC: Adding query filter params (role.id=1e04fdc0cc334fd7a733a1490a194e37, user.id=cae82ab763d546708d0c263651802760, scope.system=all) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:12.550165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a991413-4a9c-449a-99cd-400282d3542b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:12.550165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a991413-4a9c-449a-99cd-400282d3542b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:12.563957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 169/334] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Oct 4 05:24:12 2022] GET /identity/v3/role_assignments?user.id=cae82ab763d546708d0c263651802760&scope.system=all&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:14.207188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4cb3b-4e11-49c8-b212-a80171357f79 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:14.239359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4cb3b-4e11-49c8-b212-a80171357f79 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:14.239359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4cb3b-4e11-49c8-b212-a80171357f79 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:14.239359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 166/335] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:14.239629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14577a9c-70dc-4d50-b1ef-7bbcf8cd51ed None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:14.239629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14577a9c-70dc-4d50-b1ef-7bbcf8cd51ed None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:14.239629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14577a9c-70dc-4d50-b1ef-7bbcf8cd51ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:14.239629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-14577a9c-70dc-4d50-b1ef-7bbcf8cd51ed None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:14.239629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-14577a9c-70dc-4d50-b1ef-7bbcf8cd51ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:14.317144 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 170/336] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:14.460007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996183cc-643c-4763-9b41-96a350dd310f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:14.460613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996183cc-643c-4763-9b41-96a350dd310f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:14.461089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996183cc-643c-4763-9b41-96a350dd310f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:14.462907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 167/337] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:14.478765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc480ed-d552-470d-9fff-44988c52596b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:14.479043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc480ed-d552-470d-9fff-44988c52596b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:14.479324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc480ed-d552-470d-9fff-44988c52596b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:14.507089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cbc480ed-d552-470d-9fff-44988c52596b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:14.508355 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cbc480ed-d552-470d-9fff-44988c52596b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:14.588475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 171/338] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:14.844403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996183cc-643c-4763-9b41-96a350dd310f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:14.979914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996183cc-643c-4763-9b41-96a350dd310f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:15.004996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:15.021092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b0fe4-97e3-4bb5-bffb-eee87c0f2690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:15.021736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b0fe4-97e3-4bb5-bffb-eee87c0f2690 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:15.022013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b0fe4-97e3-4bb5-bffb-eee87c0f2690 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:15.022313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b0fe4-97e3-4bb5-bffb-eee87c0f2690 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:15.039010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b0fe4-97e3-4bb5-bffb-eee87c0f2690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d39b0fe4-97e3-4bb5-bffb-eee87c0f2690 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:15.043888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:15.044805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:15.045677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 168/339] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:15.052169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc480ed-d552-470d-9fff-44988c52596b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:15.056885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b0fe4-97e3-4bb5-bffb-eee87c0f2690 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:15.091545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:15.261924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b0fe4-97e3-4bb5-bffb-eee87c0f2690 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:15.288329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc480ed-d552-470d-9fff-44988c52596b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:15.333032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689a3ae-4432-4f9e-91d9-2ac8306038b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:15.333711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689a3ae-4432-4f9e-91d9-2ac8306038b6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:15.333991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689a3ae-4432-4f9e-91d9-2ac8306038b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:15.334323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689a3ae-4432-4f9e-91d9-2ac8306038b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:15.365450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689a3ae-4432-4f9e-91d9-2ac8306038b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6689a3ae-4432-4f9e-91d9-2ac8306038b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:15.370949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:15.371851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:15.372656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:15.375379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 169/340] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:24:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:15.378212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689a3ae-4432-4f9e-91d9-2ac8306038b6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:15.378512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ac4a5-a459-4a8f-95a9-65b7d7a600e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:15.379356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ac4a5-a459-4a8f-95a9-65b7d7a600e9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:15.379847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ac4a5-a459-4a8f-95a9-65b7d7a600e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:15.380321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ac4a5-a459-4a8f-95a9-65b7d7a600e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:15.381210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ac4a5-a459-4a8f-95a9-65b7d7a600e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:15.381712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ac4a5-a459-4a8f-95a9-65b7d7a600e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:15.383349 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ac4a5-a459-4a8f-95a9-65b7d7a600e9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:15.397602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 172/341] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:15.405485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ac4a5-a459-4a8f-95a9-65b7d7a600e9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:15.461970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:15.535491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:15.551081 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689a3ae-4432-4f9e-91d9-2ac8306038b6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:15.556870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ac4a5-a459-4a8f-95a9-65b7d7a600e9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:15.606527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0dc177-3e8d-4b2a-a650-4aa861709b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:15.607381 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dc177-3e8d-4b2a-a650-4aa861709b74 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:15.607826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dc177-3e8d-4b2a-a650-4aa861709b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:15.608234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0dc177-3e8d-4b2a-a650-4aa861709b74 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:15.609027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0dc177-3e8d-4b2a-a650-4aa861709b74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:15.609472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0dc177-3e8d-4b2a-a650-4aa861709b74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:15.611015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca3bd3-0923-4f36-87a2-7947b81177f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:15.611162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0dc177-3e8d-4b2a-a650-4aa861709b74 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:15.611649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca3bd3-0923-4f36-87a2-7947b81177f4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:15.612420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca3bd3-0923-4f36-87a2-7947b81177f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:15.612696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca3bd3-0923-4f36-87a2-7947b81177f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:15.621129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 170/342] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:15.622698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab998c-ef3d-47bd-ad77-e60f7b2489eb None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:15.623138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab998c-ef3d-47bd-ad77-e60f7b2489eb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:15.623549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab998c-ef3d-47bd-ad77-e60f7b2489eb None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:15.624894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 171/343] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:15.631328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514eed2-0fdd-4d84-a80a-260aa09e0b27 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:15.631807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514eed2-0fdd-4d84-a80a-260aa09e0b27 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:15.632324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514eed2-0fdd-4d84-a80a-260aa09e0b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:15.653334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a514eed2-0fdd-4d84-a80a-260aa09e0b27 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:15.655005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a514eed2-0fdd-4d84-a80a-260aa09e0b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:15.689366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca3bd3-0923-4f36-87a2-7947b81177f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:15.691458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca3bd3-0923-4f36-87a2-7947b81177f4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:15.776011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 172/344] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:15.777246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514eed2-0fdd-4d84-a80a-260aa09e0b27 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:15.780107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:15.803334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 173/345] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:24:15.812131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca3bd3-0923-4f36-87a2-7947b81177f4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:15.856501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:15.945162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca3bd3-0923-4f36-87a2-7947b81177f4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:15.962166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514eed2-0fdd-4d84-a80a-260aa09e0b27 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:15.981845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b3c9d-4477-4fe8-8580-348a3d7b4e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:15.982891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b3c9d-4477-4fe8-8580-348a3d7b4e4b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:15.983318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b3c9d-4477-4fe8-8580-348a3d7b4e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:15.983735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b3c9d-4477-4fe8-8580-348a3d7b4e4b None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:15.987754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b3c9d-4477-4fe8-8580-348a3d7b4e4b None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:15.989614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b3c9d-4477-4fe8-8580-348a3d7b4e4b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:15.991484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 174/346] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:15 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:15.999304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b3c9d-4477-4fe8-8580-348a3d7b4e4b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:16.013300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5afb93-dac5-4c50-b76a-929db9f4e897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:16.013937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5afb93-dac5-4c50-b76a-929db9f4e897 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.014282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5afb93-dac5-4c50-b76a-929db9f4e897 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.014567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5afb93-dac5-4c50-b76a-929db9f4e897 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.067493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:16.097501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5afb93-dac5-4c50-b76a-929db9f4e897 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:16.099977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5afb93-dac5-4c50-b76a-929db9f4e897 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:16.131830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b3c9d-4477-4fe8-8580-348a3d7b4e4b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:16.151781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:16.226838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79ac96-67bb-4f8b-8025-aa766246f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:16.227487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79ac96-67bb-4f8b-8025-aa766246f03c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.227766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79ac96-67bb-4f8b-8025-aa766246f03c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.228041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79ac96-67bb-4f8b-8025-aa766246f03c None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.232466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 173/347] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:16.234326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29defb-f989-40b9-8c4f-97612d547c46 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.234817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29defb-f989-40b9-8c4f-97612d547c46 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.235386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29defb-f989-40b9-8c4f-97612d547c46 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.237437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 174/348] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:16.245554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29defb-f989-40b9-8c4f-97612d547c46 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:16.245737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79ac96-67bb-4f8b-8025-aa766246f03c None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:16.245737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79ac96-67bb-4f8b-8025-aa766246f03c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:16.245737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 175/349] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:15 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:16.246585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3491a46-aea6-460c-b218-0735622aea39 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.246872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3491a46-aea6-460c-b218-0735622aea39 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.247148 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3491a46-aea6-460c-b218-0735622aea39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.280719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e3491a46-aea6-460c-b218-0735622aea39 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:16.281917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e3491a46-aea6-460c-b218-0735622aea39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:16.369120 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 176/350] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:16.370906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3491a46-aea6-460c-b218-0735622aea39 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:16.396781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29defb-f989-40b9-8c4f-97612d547c46 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:16.422670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:16.482902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e39ce-b064-43bf-b3c8-831bc606f3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:16.483573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e39ce-b064-43bf-b3c8-831bc606f3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.483898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e39ce-b064-43bf-b3c8-831bc606f3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.484117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e39ce-b064-43bf-b3c8-831bc606f3f6 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.485413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e39ce-b064-43bf-b3c8-831bc606f3f6 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:16.486907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e39ce-b064-43bf-b3c8-831bc606f3f6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:16.488979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 175/351] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:16 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:16.489628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e39ce-b064-43bf-b3c8-831bc606f3f6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:16.495766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3491a46-aea6-460c-b218-0735622aea39 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:16.498437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:16.538061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca5373-398c-4147-8735-30d087b1734a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:16.539103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca5373-398c-4147-8735-30d087b1734a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.539573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca5373-398c-4147-8735-30d087b1734a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.540050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca5373-398c-4147-8735-30d087b1734a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.541437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca5373-398c-4147-8735-30d087b1734a None admin] RBAC: Adding query filter params (group.id=4129d7a0f3ae4e28940f41fd1c1a8776, role.id=1e04fdc0cc334fd7a733a1490a194e37, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:16.541947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca5373-398c-4147-8735-30d087b1734a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:16.543621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca5373-398c-4147-8735-30d087b1734a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:16.550470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 177/352] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 05:24:16 2022] GET /identity/v3/role_assignments?group.id=4129d7a0f3ae4e28940f41fd1c1a8776&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:16.551788 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca5373-398c-4147-8735-30d087b1734a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:16.679776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca5373-398c-4147-8735-30d087b1734a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:16.688990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:16.728632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b0a96-2392-416b-aae6-d892e887c972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:16.729639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b0a96-2392-416b-aae6-d892e887c972 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.730111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b0a96-2392-416b-aae6-d892e887c972 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.730640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b0a96-2392-416b-aae6-d892e887c972 None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.732386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b0a96-2392-416b-aae6-d892e887c972 None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:16.734062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b0a96-2392-416b-aae6-d892e887c972 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:16.736114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 178/353] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:16 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:16.739008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d48ee4-a955-4cc8-b6b2-cf86164737c5 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.739547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d48ee4-a955-4cc8-b6b2-cf86164737c5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.740022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d48ee4-a955-4cc8-b6b2-cf86164737c5 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.741885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 179/354] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:16.746265 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d48ee4-a955-4cc8-b6b2-cf86164737c5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:16.766864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e39ce-b064-43bf-b3c8-831bc606f3f6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:16.824104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8ff29-d5ad-4f9a-a367-392daf3e881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:16.824831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8ff29-d5ad-4f9a-a367-392daf3e881b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.825270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8ff29-d5ad-4f9a-a367-392daf3e881b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.825551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8ff29-d5ad-4f9a-a367-392daf3e881b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.845291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8ff29-d5ad-4f9a-a367-392daf3e881b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8ad8ff29-d5ad-4f9a-a367-392daf3e881b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:16.850681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:16.851784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:16.852731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 176/355] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:16.853518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd9104-1e61-4233-9bbe-a0cc183fb345 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.853518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd9104-1e61-4233-9bbe-a0cc183fb345 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.853518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd9104-1e61-4233-9bbe-a0cc183fb345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.882646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ccfd9104-1e61-4233-9bbe-a0cc183fb345 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:16.887899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ccfd9104-1e61-4233-9bbe-a0cc183fb345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:16.917992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d48ee4-a955-4cc8-b6b2-cf86164737c5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:16.966218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf390b80-4d6c-43bd-8205-2c23e562c628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:16.966827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf390b80-4d6c-43bd-8205-2c23e562c628 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:16.967116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf390b80-4d6c-43bd-8205-2c23e562c628 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:16.967390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf390b80-4d6c-43bd-8205-2c23e562c628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:16.968163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf390b80-4d6c-43bd-8205-2c23e562c628 None admin] RBAC: Adding query filter params (group.id=4129d7a0f3ae4e28940f41fd1c1a8776, role.id=cae161e9c1b64656924448dd784ed4d1, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:16.968473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf390b80-4d6c-43bd-8205-2c23e562c628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:16.969892 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf390b80-4d6c-43bd-8205-2c23e562c628 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:16.970413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:16.978358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 180/356] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 05:24:16 2022] GET /identity/v3/role_assignments?group.id=4129d7a0f3ae4e28940f41fd1c1a8776&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=cae161e9c1b64656924448dd784ed4d1 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:16.979251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf390b80-4d6c-43bd-8205-2c23e562c628 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:16.985299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 177/357] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:16.988105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd9104-1e61-4233-9bbe-a0cc183fb345 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:17.040437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:17.129848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf390b80-4d6c-43bd-8205-2c23e562c628 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:17.194114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afec96-2d2b-4ad5-b0a9-bf4195fcdc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:17.194114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afec96-2d2b-4ad5-b0a9-bf4195fcdc33 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:17.194114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afec96-2d2b-4ad5-b0a9-bf4195fcdc33 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:17.194114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afec96-2d2b-4ad5-b0a9-bf4195fcdc33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:17.209951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afec96-2d2b-4ad5-b0a9-bf4195fcdc33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c2afec96-2d2b-4ad5-b0a9-bf4195fcdc33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:17.216613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:17.217486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 05:24:17.218460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:17.219991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 181/358] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:17.224058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cde38-cdb9-411d-8aab-6ad3b4afc4e6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:17.224058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cde38-cdb9-411d-8aab-6ad3b4afc4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:17.224058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cde38-cdb9-411d-8aab-6ad3b4afc4e6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:17.224058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 182/359] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:17.225489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cde38-cdb9-411d-8aab-6ad3b4afc4e6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:17.227748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd9104-1e61-4233-9bbe-a0cc183fb345 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:17.267851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac259636-5cfb-4f26-8024-465d03f98dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:17.268879 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac259636-5cfb-4f26-8024-465d03f98dee None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:17.269359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac259636-5cfb-4f26-8024-465d03f98dee None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:17.269829 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac259636-5cfb-4f26-8024-465d03f98dee None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:17.270774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac259636-5cfb-4f26-8024-465d03f98dee None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:17.271396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac259636-5cfb-4f26-8024-465d03f98dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:17.272967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac259636-5cfb-4f26-8024-465d03f98dee None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:17.282939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 178/360] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:17.284569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6d594-2937-442a-b959-497667d34caf None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:17.285052 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6d594-2937-442a-b959-497667d34caf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:17.285624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6d594-2937-442a-b959-497667d34caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:17.311061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-91f6d594-2937-442a-b959-497667d34caf None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:17.312850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-91f6d594-2937-442a-b959-497667d34caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:17.373735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:17.391424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 179/361] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:17.391757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:17.392359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6d594-2937-442a-b959-497667d34caf None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:17.405814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cde38-cdb9-411d-8aab-6ad3b4afc4e6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:17.469858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e91bdf-de4c-4673-89f4-470be3f798f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:17.470417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91bdf-de4c-4673-89f4-470be3f798f3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:17.470696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91bdf-de4c-4673-89f4-470be3f798f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:17.470962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91bdf-de4c-4673-89f4-470be3f798f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:17.471676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e91bdf-de4c-4673-89f4-470be3f798f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:17.472179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e91bdf-de4c-4673-89f4-470be3f798f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:17.473554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e91bdf-de4c-4673-89f4-470be3f798f3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:17.485794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 183/362] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:17.487288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e91bdf-de4c-4673-89f4-470be3f798f3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:17.548191 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6d594-2937-442a-b959-497667d34caf None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:17.620348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9a5fa-56b1-40d0-9a01-b8cac0ab5972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:17.620977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9a5fa-56b1-40d0-9a01-b8cac0ab5972 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:17.621253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9a5fa-56b1-40d0-9a01-b8cac0ab5972 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:17.621526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9a5fa-56b1-40d0-9a01-b8cac0ab5972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:17.684101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e91bdf-de4c-4673-89f4-470be3f798f3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:17.703608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:17.717393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9a5fa-56b1-40d0-9a01-b8cac0ab5972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:17.718884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9a5fa-56b1-40d0-9a01-b8cac0ab5972 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:17.769078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47ba2a-8990-490c-a715-eb2c3dcb8a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:17.769643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47ba2a-8990-490c-a715-eb2c3dcb8a17 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:17.769924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47ba2a-8990-490c-a715-eb2c3dcb8a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:17.770220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47ba2a-8990-490c-a715-eb2c3dcb8a17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:17.790435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47ba2a-8990-490c-a715-eb2c3dcb8a17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ab47ba2a-8990-490c-a715-eb2c3dcb8a17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:17.796363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:17.797251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:17.798038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:17.801770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 184/363] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:24:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:17.802837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47ba2a-8990-490c-a715-eb2c3dcb8a17 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:17.828402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:17.833982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 180/364] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:17.834892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9a5fa-56b1-40d0-9a01-b8cac0ab5972 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:17.921578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47ba2a-8990-490c-a715-eb2c3dcb8a17 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:17.928955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:17.944738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9a5fa-56b1-40d0-9a01-b8cac0ab5972 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:17.963732 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1a539-ff07-4bf0-9c11-6b334e29dae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:17.964699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1a539-ff07-4bf0-9c11-6b334e29dae8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:17.965170 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1a539-ff07-4bf0-9c11-6b334e29dae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:17.965630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1a539-ff07-4bf0-9c11-6b334e29dae8 None admin] PATH_INFO: `/v3/users/64c1e42e23694714a5c89ee380e98da5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:17.967143 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1a539-ff07-4bf0-9c11-6b334e29dae8 None admin] RBAC: Authorizing `identity:get_user(user_id=64c1e42e23694714a5c89ee380e98da5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:17.968888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1a539-ff07-4bf0-9c11-6b334e29dae8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:17.971066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 185/365] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:24:17 2022] GET /identity/v3/users/64c1e42e23694714a5c89ee380e98da5 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:17.972518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1a539-ff07-4bf0-9c11-6b334e29dae8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:17.981349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b75b2-88b0-4bd2-bceb-f1b5de3faaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:17.981998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b75b2-88b0-4bd2-bceb-f1b5de3faaea None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:17.982305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b75b2-88b0-4bd2-bceb-f1b5de3faaea None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:17.982582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b75b2-88b0-4bd2-bceb-f1b5de3faaea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:17.993295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b75b2-88b0-4bd2-bceb-f1b5de3faaea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b05b75b2-88b0-4bd2-bceb-f1b5de3faaea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:17.998175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:17.999055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:17.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 181/366] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:24:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:18.000538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b75b2-88b0-4bd2-bceb-f1b5de3faaea None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:18.083533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:18.097910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1a539-ff07-4bf0-9c11-6b334e29dae8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:18.110046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b75b2-88b0-4bd2-bceb-f1b5de3faaea None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:18.136318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa1429-9a2c-4a42-b4fb-405b55f49944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:18.137404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa1429-9a2c-4a42-b4fb-405b55f49944 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:18.137938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa1429-9a2c-4a42-b4fb-405b55f49944 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:18.138488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa1429-9a2c-4a42-b4fb-405b55f49944 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:18.139419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa1429-9a2c-4a42-b4fb-405b55f49944 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:18.140052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa1429-9a2c-4a42-b4fb-405b55f49944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:18.141786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa1429-9a2c-4a42-b4fb-405b55f49944 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:18.146160 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6a03d-9767-42d3-8b6e-b3d59f8d3302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:18.146835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6a03d-9767-42d3-8b6e-b3d59f8d3302 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:18.147116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6a03d-9767-42d3-8b6e-b3d59f8d3302 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:18.147391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6a03d-9767-42d3-8b6e-b3d59f8d3302 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:18.148619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6a03d-9767-42d3-8b6e-b3d59f8d3302 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:18.150662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6a03d-9767-42d3-8b6e-b3d59f8d3302 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:18.151867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 182/367] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:17 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:18.152645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6a03d-9767-42d3-8b6e-b3d59f8d3302 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:18.154743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 186/368] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:18.155834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa1429-9a2c-4a42-b4fb-405b55f49944 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:18.175778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:18.263984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6a03d-9767-42d3-8b6e-b3d59f8d3302 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:18.273930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa1429-9a2c-4a42-b4fb-405b55f49944 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:18.298678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca4a6a-57b3-4284-98db-326096ad708c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:18.299321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4a6a-57b3-4284-98db-326096ad708c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:18.299600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4a6a-57b3-4284-98db-326096ad708c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:18.299874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4a6a-57b3-4284-98db-326096ad708c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:18.300767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca4a6a-57b3-4284-98db-326096ad708c None admin] RBAC: Adding query filter params (role.id=840afb6280c341c6beefc211b040c463, user.id=64c1e42e23694714a5c89ee380e98da5, scope.system=all) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:18.300879 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca4a6a-57b3-4284-98db-326096ad708c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:18.302346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca4a6a-57b3-4284-98db-326096ad708c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:18.312640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915fe19-78e3-46ff-8642-1c5832a94fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:18.313514 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915fe19-78e3-46ff-8642-1c5832a94fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:18.313995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915fe19-78e3-46ff-8642-1c5832a94fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:18.314493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915fe19-78e3-46ff-8642-1c5832a94fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:18.314693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 183/369] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Oct 4 05:24:18 2022] GET /identity/v3/role_assignments?user.id=64c1e42e23694714a5c89ee380e98da5&scope.system=all&role.id=840afb6280c341c6beefc211b040c463 => generated 510 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:18.316023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca4a6a-57b3-4284-98db-326096ad708c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:18.316280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915fe19-78e3-46ff-8642-1c5832a94fc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:18.321518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915fe19-78e3-46ff-8642-1c5832a94fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:18.321518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915fe19-78e3-46ff-8642-1c5832a94fc4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:18.343503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 187/370] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:18.344386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915fe19-78e3-46ff-8642-1c5832a94fc4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:18.374674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:18.448474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:18.502174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca4a6a-57b3-4284-98db-326096ad708c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:18.506597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915fe19-78e3-46ff-8642-1c5832a94fc4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:18.544717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53617c54-0d78-428d-a38f-f61ee3b99e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:18.545677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53617c54-0d78-428d-a38f-f61ee3b99e10 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:18.546173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53617c54-0d78-428d-a38f-f61ee3b99e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:18.546640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53617c54-0d78-428d-a38f-f61ee3b99e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:18.555966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8306a-3854-4ac9-bd0f-8e85776896ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:18.556655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8306a-3854-4ac9-bd0f-8e85776896ee None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:18.556898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8306a-3854-4ac9-bd0f-8e85776896ee None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:18.557173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8306a-3854-4ac9-bd0f-8e85776896ee None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:18.558323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8306a-3854-4ac9-bd0f-8e85776896ee None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:18.559767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8306a-3854-4ac9-bd0f-8e85776896ee None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:18.561354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 184/371] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:18 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:18.562116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8306a-3854-4ac9-bd0f-8e85776896ee None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:18.627416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53617c54-0d78-428d-a38f-f61ee3b99e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:18.629864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53617c54-0d78-428d-a38f-f61ee3b99e10 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:18.645989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:18.711796 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8306a-3854-4ac9-bd0f-8e85776896ee None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:18.749078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 188/372] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:18.753806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53617c54-0d78-428d-a38f-f61ee3b99e10 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:18.774100 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffede03-2119-44d1-b678-2799a4441dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:18.774761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffede03-2119-44d1-b678-2799a4441dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:18.775036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffede03-2119-44d1-b678-2799a4441dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:18.775307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffede03-2119-44d1-b678-2799a4441dd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:18.791859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:18.870884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffede03-2119-44d1-b678-2799a4441dd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:18.872654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffede03-2119-44d1-b678-2799a4441dd6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:18.897723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53617c54-0d78-428d-a38f-f61ee3b99e10 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:18.917487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:18.950446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917f238-7772-4847-b69f-ca9b917a84bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:18.951157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917f238-7772-4847-b69f-ca9b917a84bc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:18.951588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917f238-7772-4847-b69f-ca9b917a84bc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:18.952306 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917f238-7772-4847-b69f-ca9b917a84bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:18.953181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917f238-7772-4847-b69f-ca9b917a84bc None admin] RBAC: Adding query filter params (group.id=4129d7a0f3ae4e28940f41fd1c1a8776, role.id=1e04fdc0cc334fd7a733a1490a194e37, scope.project.id=ce294031877d49d18dd64a30dbcc7c63) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:18.953498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917f238-7772-4847-b69f-ca9b917a84bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:18.955278 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917f238-7772-4847-b69f-ca9b917a84bc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:18.962746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 189/373] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 05:24:18 2022] GET /identity/v3/role_assignments?group.id=4129d7a0f3ae4e28940f41fd1c1a8776&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:18.963589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917f238-7772-4847-b69f-ca9b917a84bc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:19.065445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 185/374] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:19.071238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffede03-2119-44d1-b678-2799a4441dd6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:19.148926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:19.196410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917f238-7772-4847-b69f-ca9b917a84bc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:19.250557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffede03-2119-44d1-b678-2799a4441dd6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:19.267440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352664d1-0c8e-48fe-b77b-17ffe70f2158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:19.268153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352664d1-0c8e-48fe-b77b-17ffe70f2158 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:19.268434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352664d1-0c8e-48fe-b77b-17ffe70f2158 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:19.268711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352664d1-0c8e-48fe-b77b-17ffe70f2158 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:19.269897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352664d1-0c8e-48fe-b77b-17ffe70f2158 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:19.271401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352664d1-0c8e-48fe-b77b-17ffe70f2158 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:19.273092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 190/375] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:18 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:19.278817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352664d1-0c8e-48fe-b77b-17ffe70f2158 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:19.316518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e9bf1-4a57-4516-9d5c-c287d5e2798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:19.317171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e9bf1-4a57-4516-9d5c-c287d5e2798f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:19.317454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e9bf1-4a57-4516-9d5c-c287d5e2798f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:19.317730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e9bf1-4a57-4516-9d5c-c287d5e2798f None admin] PATH_INFO: `/v3/projects/4c2b6383659b48fcabc56888b6bfb190` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:19.319359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e9bf1-4a57-4516-9d5c-c287d5e2798f None admin] RBAC: Authorizing `identity:get_project(project_id=4c2b6383659b48fcabc56888b6bfb190)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:19.320821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e9bf1-4a57-4516-9d5c-c287d5e2798f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:19.323319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 186/376] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:19 2022] GET /identity/v3/projects/4c2b6383659b48fcabc56888b6bfb190 => generated 344 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:19.331201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e9bf1-4a57-4516-9d5c-c287d5e2798f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:19.350914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:19.415164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352664d1-0c8e-48fe-b77b-17ffe70f2158 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:19.453002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77644dfd-94a4-45bd-9765-c5a8f080c982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:19.453813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77644dfd-94a4-45bd-9765-c5a8f080c982 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:19.454241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77644dfd-94a4-45bd-9765-c5a8f080c982 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:19.454618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77644dfd-94a4-45bd-9765-c5a8f080c982 None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:19.456440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77644dfd-94a4-45bd-9765-c5a8f080c982 None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:19.457463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e9bf1-4a57-4516-9d5c-c287d5e2798f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:19.458097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77644dfd-94a4-45bd-9765-c5a8f080c982 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:19.460144 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 191/377] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:19 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:19.461428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d37e36-89ab-44f2-a230-21322512bc1f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:19.461810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d37e36-89ab-44f2-a230-21322512bc1f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:19.462215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d37e36-89ab-44f2-a230-21322512bc1f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:19.463818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 192/378] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:19.469165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d37e36-89ab-44f2-a230-21322512bc1f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:19.470183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:19.495506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52526043-2254-47b4-843b-f928759cbd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:19.496154 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52526043-2254-47b4-843b-f928759cbd0a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:19.496432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52526043-2254-47b4-843b-f928759cbd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:19.496707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52526043-2254-47b4-843b-f928759cbd0a None admin] PATH_INFO: `/v3/groups/322918b086f34f49a61bb04617ebcdc9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:19.503672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52526043-2254-47b4-843b-f928759cbd0a None admin] RBAC: Authorizing `identity:get_group(group_id=322918b086f34f49a61bb04617ebcdc9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:19.505226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52526043-2254-47b4-843b-f928759cbd0a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:19.512036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 187/379] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:19 2022] GET /identity/v3/groups/322918b086f34f49a61bb04617ebcdc9 => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:19.513705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb9833-aada-4be3-a5e3-75dbeab3083f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:19.514248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb9833-aada-4be3-a5e3-75dbeab3083f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:19.514661 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb9833-aada-4be3-a5e3-75dbeab3083f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:19.538170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-21eb9833-aada-4be3-a5e3-75dbeab3083f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:19.539503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-21eb9833-aada-4be3-a5e3-75dbeab3083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:19.618646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d37e36-89ab-44f2-a230-21322512bc1f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:19.626455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 188/380] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:19.627310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb9833-aada-4be3-a5e3-75dbeab3083f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:19.643554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:19.677447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d9881-e876-4d0b-857c-41e33f4f13f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:19.678542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d9881-e876-4d0b-857c-41e33f4f13f3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:19.679165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d9881-e876-4d0b-857c-41e33f4f13f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:19.679864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d9881-e876-4d0b-857c-41e33f4f13f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:19.680893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d9881-e876-4d0b-857c-41e33f4f13f3 None admin] RBAC: Adding query filter params (group.id=4129d7a0f3ae4e28940f41fd1c1a8776, role.id=cae161e9c1b64656924448dd784ed4d1, scope.project.id=ce294031877d49d18dd64a30dbcc7c63) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:19.681158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d9881-e876-4d0b-857c-41e33f4f13f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:19.682914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d9881-e876-4d0b-857c-41e33f4f13f3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:19.691976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 193/381] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 05:24:19 2022] GET /identity/v3/role_assignments?group.id=4129d7a0f3ae4e28940f41fd1c1a8776&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=cae161e9c1b64656924448dd784ed4d1 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:19.693278 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e82c27-4b66-489d-b2bc-ffb56482a645 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:19.693898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e82c27-4b66-489d-b2bc-ffb56482a645 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:19.694288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e82c27-4b66-489d-b2bc-ffb56482a645 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:19.702219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 194/382] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:19.703213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13b668-8bb8-4989-bf28-e0b70b532276 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:19.703798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13b668-8bb8-4989-bf28-e0b70b532276 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:19.705057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13b668-8bb8-4989-bf28-e0b70b532276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:19.731307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9e13b668-8bb8-4989-bf28-e0b70b532276 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:19.732630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9e13b668-8bb8-4989-bf28-e0b70b532276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:19.767163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:19.772231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb9833-aada-4be3-a5e3-75dbeab3083f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:19.821915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c130bd-8361-41ea-b828-5f058c3fb75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:19.822866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c130bd-8361-41ea-b828-5f058c3fb75f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:19.823313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c130bd-8361-41ea-b828-5f058c3fb75f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:19.823790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c130bd-8361-41ea-b828-5f058c3fb75f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:19.824697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c130bd-8361-41ea-b828-5f058c3fb75f None admin] RBAC: Adding query filter params (group.id=322918b086f34f49a61bb04617ebcdc9, role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=4c2b6383659b48fcabc56888b6bfb190) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:19.825180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c130bd-8361-41ea-b828-5f058c3fb75f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:19.829419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c130bd-8361-41ea-b828-5f058c3fb75f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:19.832664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 195/383] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:19.836634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 189/384] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 05:24:19 2022] GET /identity/v3/role_assignments?group.id=322918b086f34f49a61bb04617ebcdc9&scope.project.id=4c2b6383659b48fcabc56888b6bfb190&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:19.936628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13b668-8bb8-4989-bf28-e0b70b532276 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:20.050339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13b668-8bb8-4989-bf28-e0b70b532276 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:20.058527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:20.085869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40317edf-bdfe-482e-a939-7f74ad886448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:20.090300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317edf-bdfe-482e-a939-7f74ad886448 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:20.090300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317edf-bdfe-482e-a939-7f74ad886448 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:20.090300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317edf-bdfe-482e-a939-7f74ad886448 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:20.098030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40317edf-bdfe-482e-a939-7f74ad886448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-40317edf-bdfe-482e-a939-7f74ad886448 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:20.103079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:20.103959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:20.104840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 196/385] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:20.111496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c130bd-8361-41ea-b828-5f058c3fb75f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:20.135179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40317edf-bdfe-482e-a939-7f74ad886448 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:20.245751 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40317edf-bdfe-482e-a939-7f74ad886448 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:20.277510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c130bd-8361-41ea-b828-5f058c3fb75f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:20.282057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bbed4-51ec-4b36-bbe1-cdf5c3fc6508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:20.282728 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bbed4-51ec-4b36-bbe1-cdf5c3fc6508 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:20.283008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bbed4-51ec-4b36-bbe1-cdf5c3fc6508 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:20.283281 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bbed4-51ec-4b36-bbe1-cdf5c3fc6508 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:20.295200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563bbed4-51ec-4b36-bbe1-cdf5c3fc6508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-563bbed4-51ec-4b36-bbe1-cdf5c3fc6508 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:20.300392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:20.301254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:20.302699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 197/386] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:24:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:20.306918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bbed4-51ec-4b36-bbe1-cdf5c3fc6508 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:20.348506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221bf57-63b7-4246-be7b-eeeeba41cb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:20.349126 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221bf57-63b7-4246-be7b-eeeeba41cb99 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:20.349405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221bf57-63b7-4246-be7b-eeeeba41cb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:20.349720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221bf57-63b7-4246-be7b-eeeeba41cb99 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:20.350369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221bf57-63b7-4246-be7b-eeeeba41cb99 None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:20.350670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221bf57-63b7-4246-be7b-eeeeba41cb99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:20.352069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221bf57-63b7-4246-be7b-eeeeba41cb99 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:20.362556 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:20.376262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 190/387] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:20.399812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221bf57-63b7-4246-be7b-eeeeba41cb99 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:20.450508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:20.499492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bbed4-51ec-4b36-bbe1-cdf5c3fc6508 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:20.567819 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaeafa-8872-4af9-a7a9-12eb186f5554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:20.568687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaeafa-8872-4af9-a7a9-12eb186f5554 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:20.568976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaeafa-8872-4af9-a7a9-12eb186f5554 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:20.569242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaeafa-8872-4af9-a7a9-12eb186f5554 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:20.569907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaeafa-8872-4af9-a7a9-12eb186f5554 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:20.570231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaeafa-8872-4af9-a7a9-12eb186f5554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:20.571605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaeafa-8872-4af9-a7a9-12eb186f5554 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:20.576750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221bf57-63b7-4246-be7b-eeeeba41cb99 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:20.592543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 198/388] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:20.598703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaeafa-8872-4af9-a7a9-12eb186f5554 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:20.641519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42ea0d-ad73-4464-a55d-15695bf29984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:20.642207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42ea0d-ad73-4464-a55d-15695bf29984 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:20.642454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42ea0d-ad73-4464-a55d-15695bf29984 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:20.642724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42ea0d-ad73-4464-a55d-15695bf29984 None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:20.644509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42ea0d-ad73-4464-a55d-15695bf29984 None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:20.645945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42ea0d-ad73-4464-a55d-15695bf29984 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:20.647536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 191/389] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:20 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:20.654581 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42ea0d-ad73-4464-a55d-15695bf29984 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:20.774799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:20.808326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaeafa-8872-4af9-a7a9-12eb186f5554 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:20.813507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42ea0d-ad73-4464-a55d-15695bf29984 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:20.816871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:20.863129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03925ad-ef1e-4990-9c65-9e4a18865082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:20.864262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03925ad-ef1e-4990-9c65-9e4a18865082 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:20.864546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03925ad-ef1e-4990-9c65-9e4a18865082 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:20.864823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03925ad-ef1e-4990-9c65-9e4a18865082 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:20.867775 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03925ad-ef1e-4990-9c65-9e4a18865082 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:20.869225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03925ad-ef1e-4990-9c65-9e4a18865082 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:20.874648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 192/390] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:20 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:20.879620 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3a900-ca95-43ac-a3f2-8cb468eb2483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:20.881072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03925ad-ef1e-4990-9c65-9e4a18865082 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:20.881871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3a900-ca95-43ac-a3f2-8cb468eb2483 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:20.882183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3a900-ca95-43ac-a3f2-8cb468eb2483 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:20.882465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3a900-ca95-43ac-a3f2-8cb468eb2483 None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:20.898646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3a900-ca95-43ac-a3f2-8cb468eb2483 None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:20.900125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3a900-ca95-43ac-a3f2-8cb468eb2483 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:20.903319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 199/391] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:20 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:20.908848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3a900-ca95-43ac-a3f2-8cb468eb2483 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:21.092192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03925ad-ef1e-4990-9c65-9e4a18865082 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:21.120053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3a900-ca95-43ac-a3f2-8cb468eb2483 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:21.134675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ad1ac-e61f-4711-a487-d4db1b6ec60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:21.135644 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ad1ac-e61f-4711-a487-d4db1b6ec60d None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:21.135951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ad1ac-e61f-4711-a487-d4db1b6ec60d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:21.136369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ad1ac-e61f-4711-a487-d4db1b6ec60d None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/groups/4129d7a0f3ae4e28940f41fd1c1a8776/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:21.138697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ad1ac-e61f-4711-a487-d4db1b6ec60d None admin] RBAC: Authorizing `identity:create_grant(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9, group_id=4129d7a0f3ae4e28940f41fd1c1a8776, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:21.140667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ad1ac-e61f-4711-a487-d4db1b6ec60d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:21.169709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2f261-c06c-4324-89f2-e7f33f4154ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:21.170406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2f261-c06c-4324-89f2-e7f33f4154ba None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:21.170741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2f261-c06c-4324-89f2-e7f33f4154ba None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:21.171024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2f261-c06c-4324-89f2-e7f33f4154ba None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:21.172159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2f261-c06c-4324-89f2-e7f33f4154ba None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:21.173653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2f261-c06c-4324-89f2-e7f33f4154ba None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:21.176365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 200/392] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:20 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:21.181726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2f261-c06c-4324-89f2-e7f33f4154ba None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:21.211050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:21.224989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 193/393] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Oct 4 05:24:20 2022] PUT /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/groups/4129d7a0f3ae4e28940f41fd1c1a8776/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:24:21.296397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2f261-c06c-4324-89f2-e7f33f4154ba None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:21.334640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998e1bc-175f-4014-9548-a1ea8c177169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:21.335314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998e1bc-175f-4014-9548-a1ea8c177169 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:21.335595 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998e1bc-175f-4014-9548-a1ea8c177169 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:21.335868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998e1bc-175f-4014-9548-a1ea8c177169 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/groups/4129d7a0f3ae4e28940f41fd1c1a8776/roles/cae161e9c1b64656924448dd784ed4d1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:21.338114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998e1bc-175f-4014-9548-a1ea8c177169 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9, group_id=4129d7a0f3ae4e28940f41fd1c1a8776, role_id=cae161e9c1b64656924448dd784ed4d1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:21.339614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998e1bc-175f-4014-9548-a1ea8c177169 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:21.427932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 201/394] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Oct 4 05:24:21 2022] PUT /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9/groups/4129d7a0f3ae4e28940f41fd1c1a8776/roles/cae161e9c1b64656924448dd784ed4d1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:24:21.899795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99548b7-d5e6-4c16-a551-6b6d064a8edb None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:21.900231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99548b7-d5e6-4c16-a551-6b6d064a8edb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:21.900422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99548b7-d5e6-4c16-a551-6b6d064a8edb None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:21.902107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 194/395] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:21.907582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e3d2c-b41b-4fdb-979d-a77f8687fb6c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:21.908014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e3d2c-b41b-4fdb-979d-a77f8687fb6c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:21.908205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e3d2c-b41b-4fdb-979d-a77f8687fb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:21.928835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-505e3d2c-b41b-4fdb-979d-a77f8687fb6c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:21.930174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-505e3d2c-b41b-4fdb-979d-a77f8687fb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:22.015833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 202/396] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:22.366443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99548b7-d5e6-4c16-a551-6b6d064a8edb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:22.397143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:22.498427 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99548b7-d5e6-4c16-a551-6b6d064a8edb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:22.540451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e917b5-f15a-4886-ab22-429df5788e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:22.541096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e917b5-f15a-4886-ab22-429df5788e6e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:22.541375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e917b5-f15a-4886-ab22-429df5788e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:22.541644 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e917b5-f15a-4886-ab22-429df5788e6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:22.555669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e917b5-f15a-4886-ab22-429df5788e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a9e917b5-f15a-4886-ab22-429df5788e6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:22.560601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:22.561534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:22.562401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:22.563043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 195/397] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:22.568761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e3d2c-b41b-4fdb-979d-a77f8687fb6c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:22.584567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:22.597904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4bd5-2dbe-4af5-97c2-7477de964177 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:22.598264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4bd5-2dbe-4af5-97c2-7477de964177 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:22.598479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4bd5-2dbe-4af5-97c2-7477de964177 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:22.599832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 196/398] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:24:22.607010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c98ae9-54dd-49d8-a0e3-7c5d17ed3317 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:22.607756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c98ae9-54dd-49d8-a0e3-7c5d17ed3317 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:22.608206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c98ae9-54dd-49d8-a0e3-7c5d17ed3317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:22.631196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d5c98ae9-54dd-49d8-a0e3-7c5d17ed3317 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:22.632709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d5c98ae9-54dd-49d8-a0e3-7c5d17ed3317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:22.694699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e3d2c-b41b-4fdb-979d-a77f8687fb6c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:22.728513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 197/399] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:22.730739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b843aa7-657a-431b-a2e1-46d620ba533f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:22.731290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b843aa7-657a-431b-a2e1-46d620ba533f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:22.731859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b843aa7-657a-431b-a2e1-46d620ba533f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:22.738761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 198/400] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:22.739691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10300008-f3c9-4b2f-b3ae-da44431bf0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:22.740386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10300008-f3c9-4b2f-b3ae-da44431bf0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:22.740737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10300008-f3c9-4b2f-b3ae-da44431bf0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:22.741086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10300008-f3c9-4b2f-b3ae-da44431bf0e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:22.741828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10300008-f3c9-4b2f-b3ae-da44431bf0e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:22.745455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10300008-f3c9-4b2f-b3ae-da44431bf0e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:22.746957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62094a-b0f9-4ab8-ad3f-29a417d9ec45 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:22.747067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10300008-f3c9-4b2f-b3ae-da44431bf0e8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:22.747555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62094a-b0f9-4ab8-ad3f-29a417d9ec45 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:22.748004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62094a-b0f9-4ab8-ad3f-29a417d9ec45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:22.757277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 203/401] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:22.762577 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10300008-f3c9-4b2f-b3ae-da44431bf0e8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:22.776715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-be62094a-b0f9-4ab8-ad3f-29a417d9ec45 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:22.779104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-be62094a-b0f9-4ab8-ad3f-29a417d9ec45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:22.792955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:22.894847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:22.910692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 199/402] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:22.934841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10300008-f3c9-4b2f-b3ae-da44431bf0e8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:22.983996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248f497-2b38-462a-8c2c-a1772acd8c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:22.984625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248f497-2b38-462a-8c2c-a1772acd8c25 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:22.984903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248f497-2b38-462a-8c2c-a1772acd8c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:22.985178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248f497-2b38-462a-8c2c-a1772acd8c25 None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:22.990787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248f497-2b38-462a-8c2c-a1772acd8c25 None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:22.992219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248f497-2b38-462a-8c2c-a1772acd8c25 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:22.994079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 204/403] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:22 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:22.999370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62094a-b0f9-4ab8-ad3f-29a417d9ec45 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:23.116953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62094a-b0f9-4ab8-ad3f-29a417d9ec45 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:23.153498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b468a-4338-4be5-aa0e-9705fde154de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:23.154037 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b468a-4338-4be5-aa0e-9705fde154de None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:23.154340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b468a-4338-4be5-aa0e-9705fde154de None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:23.154608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b468a-4338-4be5-aa0e-9705fde154de None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:23.155730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b468a-4338-4be5-aa0e-9705fde154de None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:23.157341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b468a-4338-4be5-aa0e-9705fde154de None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:23.159640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 200/404] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:22 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:23.165358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248f497-2b38-462a-8c2c-a1772acd8c25 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:23.169848 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b468a-4338-4be5-aa0e-9705fde154de None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:23.199170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:23.329241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248f497-2b38-462a-8c2c-a1772acd8c25 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:23.331007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b468a-4338-4be5-aa0e-9705fde154de None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:23.375866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a3326-b5b8-478f-be5e-a8ac99202dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:23.376516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a3326-b5b8-478f-be5e-a8ac99202dcb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:23.376796 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a3326-b5b8-478f-be5e-a8ac99202dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:23.377143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a3326-b5b8-478f-be5e-a8ac99202dcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:23.378879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbf4dd-a417-4aae-ab56-540332a679d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:23.379839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbf4dd-a417-4aae-ab56-540332a679d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:23.380542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbf4dd-a417-4aae-ab56-540332a679d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:23.380819 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbf4dd-a417-4aae-ab56-540332a679d3 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63/groups/4129d7a0f3ae4e28940f41fd1c1a8776/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:23.383462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbf4dd-a417-4aae-ab56-540332a679d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce294031877d49d18dd64a30dbcc7c63, group_id=4129d7a0f3ae4e28940f41fd1c1a8776, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:23.384924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbf4dd-a417-4aae-ab56-540332a679d3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:23.388770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a3326-b5b8-478f-be5e-a8ac99202dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-654a3326-b5b8-478f-be5e-a8ac99202dcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:23.394451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:23.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:23.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 201/405] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:24:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:23.397457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a3326-b5b8-478f-be5e-a8ac99202dcb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:23.478768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:23.489595 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 205/406] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Oct 4 05:24:23 2022] PUT /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63/groups/4129d7a0f3ae4e28940f41fd1c1a8776/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:23.494478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbf4dd-a417-4aae-ab56-540332a679d3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:23.501461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:23.524022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a3326-b5b8-478f-be5e-a8ac99202dcb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:23.591019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640cec6-ef83-41fa-bb0d-a3046fe47238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:23.592099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640cec6-ef83-41fa-bb0d-a3046fe47238 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:23.593964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640cec6-ef83-41fa-bb0d-a3046fe47238 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:23.594510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640cec6-ef83-41fa-bb0d-a3046fe47238 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:23.621287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640cec6-ef83-41fa-bb0d-a3046fe47238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b640cec6-ef83-41fa-bb0d-a3046fe47238 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:23.634532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:23.635505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:23.636281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:23.640439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 202/407] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:24:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:23.646994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640cec6-ef83-41fa-bb0d-a3046fe47238 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:23.659986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbf4dd-a417-4aae-ab56-540332a679d3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:23.736065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5fda0-7734-4201-a4ad-124ddd8d4861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:23.736661 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5fda0-7734-4201-a4ad-124ddd8d4861 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:23.737020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5fda0-7734-4201-a4ad-124ddd8d4861 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:23.737216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5fda0-7734-4201-a4ad-124ddd8d4861 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:23.737902 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5fda0-7734-4201-a4ad-124ddd8d4861 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:23.738246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5fda0-7734-4201-a4ad-124ddd8d4861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:23.739797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5fda0-7734-4201-a4ad-124ddd8d4861 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:23.753963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 206/408] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:23.761142 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5fda0-7734-4201-a4ad-124ddd8d4861 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:23.804285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640cec6-ef83-41fa-bb0d-a3046fe47238 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:23.834948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:23.840868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:23.842069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286685d-0ab6-4a0d-aa22-73beaa92a11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:23.856582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286685d-0ab6-4a0d-aa22-73beaa92a11e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:23.856582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286685d-0ab6-4a0d-aa22-73beaa92a11e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:23.856582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286685d-0ab6-4a0d-aa22-73beaa92a11e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:23.856582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286685d-0ab6-4a0d-aa22-73beaa92a11e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:23.856582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286685d-0ab6-4a0d-aa22-73beaa92a11e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:23.856582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286685d-0ab6-4a0d-aa22-73beaa92a11e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:23.856992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 203/409] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:23.860842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286685d-0ab6-4a0d-aa22-73beaa92a11e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:23.879139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5fda0-7734-4201-a4ad-124ddd8d4861 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:23.924994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce464ed-dc7f-4398-9092-17589c1c0b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:23.925965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce464ed-dc7f-4398-9092-17589c1c0b8f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:23.926444 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce464ed-dc7f-4398-9092-17589c1c0b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:23.926901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce464ed-dc7f-4398-9092-17589c1c0b8f None admin] PATH_INFO: `/v3/groups/322918b086f34f49a61bb04617ebcdc9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:23.931697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce464ed-dc7f-4398-9092-17589c1c0b8f None admin] RBAC: Authorizing `identity:get_group(group_id=322918b086f34f49a61bb04617ebcdc9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:23.940246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce464ed-dc7f-4398-9092-17589c1c0b8f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:23.940246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 207/410] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:23 2022] GET /identity/v3/groups/322918b086f34f49a61bb04617ebcdc9 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:23.941965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce464ed-dc7f-4398-9092-17589c1c0b8f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:23.987483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286685d-0ab6-4a0d-aa22-73beaa92a11e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:24.022479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d449fb-0b67-4539-a742-a5ab491501af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:24.029345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d449fb-0b67-4539-a742-a5ab491501af None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:24.029683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d449fb-0b67-4539-a742-a5ab491501af None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:24.029971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d449fb-0b67-4539-a742-a5ab491501af None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:24.031340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d449fb-0b67-4539-a742-a5ab491501af None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:24.032920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d449fb-0b67-4539-a742-a5ab491501af None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:24.042576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 204/411] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:23 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:24.043690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d449fb-0b67-4539-a742-a5ab491501af None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:24.088404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce464ed-dc7f-4398-9092-17589c1c0b8f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:24.128424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fefd46-0922-42eb-98c1-d5a5483ae8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:24.129080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fefd46-0922-42eb-98c1-d5a5483ae8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:24.129358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fefd46-0922-42eb-98c1-d5a5483ae8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:24.129629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fefd46-0922-42eb-98c1-d5a5483ae8a0 None admin] PATH_INFO: `/v3/projects/4c2b6383659b48fcabc56888b6bfb190` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:24.133115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fefd46-0922-42eb-98c1-d5a5483ae8a0 None admin] RBAC: Authorizing `identity:get_project(project_id=4c2b6383659b48fcabc56888b6bfb190)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:24.135504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fefd46-0922-42eb-98c1-d5a5483ae8a0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:24.139985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 208/412] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:23 2022] GET /identity/v3/projects/4c2b6383659b48fcabc56888b6bfb190 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:24.145249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fefd46-0922-42eb-98c1-d5a5483ae8a0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:24.148478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:24.158546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:24.219260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d449fb-0b67-4539-a742-a5ab491501af None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:24.265973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f58800-afc6-463a-99e4-c0c581af688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:24.266945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58800-afc6-463a-99e4-c0c581af688d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:24.267410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58800-afc6-463a-99e4-c0c581af688d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:24.268034 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58800-afc6-463a-99e4-c0c581af688d None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:24.270156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f58800-afc6-463a-99e4-c0c581af688d None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:24.271929 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f58800-afc6-463a-99e4-c0c581af688d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:24.274161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 205/413] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:24 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:24.275466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca67df9-d73a-4ed0-8923-087846a23f59 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:24.275868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca67df9-d73a-4ed0-8923-087846a23f59 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:24.276285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca67df9-d73a-4ed0-8923-087846a23f59 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:24.277616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 206/414] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:24.279551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca67df9-d73a-4ed0-8923-087846a23f59 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:24.288935 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fefd46-0922-42eb-98c1-d5a5483ae8a0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:24.331552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d20ec-c87c-46e2-b1f4-098dbbec8a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:24.332215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d20ec-c87c-46e2-b1f4-098dbbec8a35 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:24.332496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d20ec-c87c-46e2-b1f4-098dbbec8a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:24.332774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d20ec-c87c-46e2-b1f4-098dbbec8a35 None admin] PATH_INFO: `/v3/projects/4c2b6383659b48fcabc56888b6bfb190/groups/322918b086f34f49a61bb04617ebcdc9/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:24.335297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d20ec-c87c-46e2-b1f4-098dbbec8a35 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c2b6383659b48fcabc56888b6bfb190, group_id=322918b086f34f49a61bb04617ebcdc9, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:24.336724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d20ec-c87c-46e2-b1f4-098dbbec8a35 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:24.385130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca67df9-d73a-4ed0-8923-087846a23f59 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:24.412102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:24.415296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 209/415] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Oct 4 05:24:24 2022] PUT /identity/v3/projects/4c2b6383659b48fcabc56888b6bfb190/groups/322918b086f34f49a61bb04617ebcdc9/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:24.416823 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d4294-7129-4ae7-ac5d-e30c7cb6c66d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:24.417186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d4294-7129-4ae7-ac5d-e30c7cb6c66d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:24.417526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d4294-7129-4ae7-ac5d-e30c7cb6c66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:24.426176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c0a84-d348-49cf-a84c-0b992ce442e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:24.426176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0a84-d348-49cf-a84c-0b992ce442e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:24.426494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0a84-d348-49cf-a84c-0b992ce442e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:24.426703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0a84-d348-49cf-a84c-0b992ce442e5 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63/groups/4129d7a0f3ae4e28940f41fd1c1a8776/roles/cae161e9c1b64656924448dd784ed4d1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:24.434753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0a84-d348-49cf-a84c-0b992ce442e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce294031877d49d18dd64a30dbcc7c63, group_id=4129d7a0f3ae4e28940f41fd1c1a8776, role_id=cae161e9c1b64656924448dd784ed4d1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:24.434753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0a84-d348-49cf-a84c-0b992ce442e5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:24.443007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8f2d4294-7129-4ae7-ac5d-e30c7cb6c66d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:24.444518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8f2d4294-7129-4ae7-ac5d-e30c7cb6c66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:24.514964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:24.524580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 207/416] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Oct 4 05:24:24 2022] PUT /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63/groups/4129d7a0f3ae4e28940f41fd1c1a8776/roles/cae161e9c1b64656924448dd784ed4d1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:24.526355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8255-a36c-44b9-8957-3a050c1687a3 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:24.526793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8255-a36c-44b9-8957-3a050c1687a3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:24.527079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8255-a36c-44b9-8957-3a050c1687a3 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:24.539512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 208/417] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:24.543282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148b6c2-7415-44ac-878b-4f3f6b0af892 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:24.543755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148b6c2-7415-44ac-878b-4f3f6b0af892 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:24.544034 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148b6c2-7415-44ac-878b-4f3f6b0af892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:24.550451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 210/418] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:24.586262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6148b6c2-7415-44ac-878b-4f3f6b0af892 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:24.587659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6148b6c2-7415-44ac-878b-4f3f6b0af892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:24.702427 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 209/419] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:24.825825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d4294-7129-4ae7-ac5d-e30c7cb6c66d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:24.971635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d4294-7129-4ae7-ac5d-e30c7cb6c66d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:25.019686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18fe6a-38a1-4b55-a655-e22123718a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:25.020700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18fe6a-38a1-4b55-a655-e22123718a97 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:25.021211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18fe6a-38a1-4b55-a655-e22123718a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:25.021772 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18fe6a-38a1-4b55-a655-e22123718a97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:25.034387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148b6c2-7415-44ac-878b-4f3f6b0af892 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:25.043061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18fe6a-38a1-4b55-a655-e22123718a97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6a18fe6a-38a1-4b55-a655-e22123718a97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:25.048282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:25.049277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:25.050096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:25.052220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 211/420] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:25.057899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18fe6a-38a1-4b55-a655-e22123718a97 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:25.155812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:25.208267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148b6c2-7415-44ac-878b-4f3f6b0af892 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:25.213297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18fe6a-38a1-4b55-a655-e22123718a97 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:25.250487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:25.259004 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfae635-440c-4c3f-bcf6-141f60c70ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:25.260052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfae635-440c-4c3f-bcf6-141f60c70ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:25.261624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfae635-440c-4c3f-bcf6-141f60c70ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:25.262050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfae635-440c-4c3f-bcf6-141f60c70ec1 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:25.263838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfae635-440c-4c3f-bcf6-141f60c70ec1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:25.264918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfae635-440c-4c3f-bcf6-141f60c70ec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:25.266704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfae635-440c-4c3f-bcf6-141f60c70ec1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:25.275937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359b9c5-42a6-475e-9cb4-7d6d6e0f2257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:25.276770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359b9c5-42a6-475e-9cb4-7d6d6e0f2257 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:25.277289 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359b9c5-42a6-475e-9cb4-7d6d6e0f2257 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:25.277701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359b9c5-42a6-475e-9cb4-7d6d6e0f2257 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:25.286216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 212/421] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:25.291881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfae635-440c-4c3f-bcf6-141f60c70ec1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:25.296871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359b9c5-42a6-475e-9cb4-7d6d6e0f2257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c359b9c5-42a6-475e-9cb4-7d6d6e0f2257 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:25.307258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:25.309555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:25.310582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:25.314425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 210/422] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:24:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:25.321055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359b9c5-42a6-475e-9cb4-7d6d6e0f2257 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:25.462770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfae635-440c-4c3f-bcf6-141f60c70ec1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:25.488519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:25.522882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359b9c5-42a6-475e-9cb4-7d6d6e0f2257 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:25.527839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938aced-5b38-4997-9b73-630ab10b6601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:25.528671 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938aced-5b38-4997-9b73-630ab10b6601 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:25.529966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938aced-5b38-4997-9b73-630ab10b6601 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:25.530518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938aced-5b38-4997-9b73-630ab10b6601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:25.587362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa163ff-ca18-469c-8e31-7835fc349994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:25.588955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa163ff-ca18-469c-8e31-7835fc349994 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:25.589412 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa163ff-ca18-469c-8e31-7835fc349994 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:25.590038 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa163ff-ca18-469c-8e31-7835fc349994 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:25.590945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa163ff-ca18-469c-8e31-7835fc349994 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:25.593528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa163ff-ca18-469c-8e31-7835fc349994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:25.595199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa163ff-ca18-469c-8e31-7835fc349994 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:25.616204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 211/423] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:25.626269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa163ff-ca18-469c-8e31-7835fc349994 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:25.663272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938aced-5b38-4997-9b73-630ab10b6601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:25.664723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938aced-5b38-4997-9b73-630ab10b6601 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:25.703039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:25.767449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa163ff-ca18-469c-8e31-7835fc349994 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:25.822354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05538e48-a8ab-4f1d-925d-967ba5e838fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:25.823265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05538e48-a8ab-4f1d-925d-967ba5e838fc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:25.823546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05538e48-a8ab-4f1d-925d-967ba5e838fc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:25.823832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05538e48-a8ab-4f1d-925d-967ba5e838fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:25.890918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 213/424] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:24:25.896457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938aced-5b38-4997-9b73-630ab10b6601 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:25.906742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05538e48-a8ab-4f1d-925d-967ba5e838fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:25.908815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05538e48-a8ab-4f1d-925d-967ba5e838fc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:25.925977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:25.957882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:26.058673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 212/425] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:26.065182 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05538e48-a8ab-4f1d-925d-967ba5e838fc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:26.091285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938aced-5b38-4997-9b73-630ab10b6601 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:26.163683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de8c97-4911-446a-b98f-a58aa55a2cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:26.164322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de8c97-4911-446a-b98f-a58aa55a2cae None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:26.164597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de8c97-4911-446a-b98f-a58aa55a2cae None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:26.164860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de8c97-4911-446a-b98f-a58aa55a2cae None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:26.166103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de8c97-4911-446a-b98f-a58aa55a2cae None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:26.167535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de8c97-4911-446a-b98f-a58aa55a2cae None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:26.169298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 214/426] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:25 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:26.175190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de8c97-4911-446a-b98f-a58aa55a2cae None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:26.228464 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05538e48-a8ab-4f1d-925d-967ba5e838fc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:26.287993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81a91d-ff85-4e00-a64a-f7e6bd496294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:26.288855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a91d-ff85-4e00-a64a-f7e6bd496294 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:26.289695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a91d-ff85-4e00-a64a-f7e6bd496294 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:26.289976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81a91d-ff85-4e00-a64a-f7e6bd496294 None admin] PATH_INFO: `/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:26.291362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a91d-ff85-4e00-a64a-f7e6bd496294 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d3c9e097bf4d3ea90c631eadb9d0d9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:26.293883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81a91d-ff85-4e00-a64a-f7e6bd496294 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:26.295899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 213/427] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:26 2022] GET /identity/v3/projects/a0d3c9e097bf4d3ea90c631eadb9d0d9 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:26.297117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780b5b8-3888-4e94-9b20-23ced9de46ca None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:26.297402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780b5b8-3888-4e94-9b20-23ced9de46ca None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:26.297669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780b5b8-3888-4e94-9b20-23ced9de46ca None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:26.299676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 214/428] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:26.301093 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780b5b8-3888-4e94-9b20-23ced9de46ca None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:26.307705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:26.321345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:26.331460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de8c97-4911-446a-b98f-a58aa55a2cae None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:26.376218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df132d6-72b4-42f9-8e5d-64c6d18f2509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:26.377823 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df132d6-72b4-42f9-8e5d-64c6d18f2509 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:26.378500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df132d6-72b4-42f9-8e5d-64c6d18f2509 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:26.379358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df132d6-72b4-42f9-8e5d-64c6d18f2509 None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:26.380919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df132d6-72b4-42f9-8e5d-64c6d18f2509 None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:26.383433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df132d6-72b4-42f9-8e5d-64c6d18f2509 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:26.386286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 215/429] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:26 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:26.387643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d001e4-07cf-4f3a-ac36-9a639d14530e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:26.388096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d001e4-07cf-4f3a-ac36-9a639d14530e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:26.388548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d001e4-07cf-4f3a-ac36-9a639d14530e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:26.411460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-30d001e4-07cf-4f3a-ac36-9a639d14530e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:26.413585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-30d001e4-07cf-4f3a-ac36-9a639d14530e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:26.441296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780b5b8-3888-4e94-9b20-23ced9de46ca None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:26.488615 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9a239-8b13-49ee-983c-f2b730212075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:26.489255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9a239-8b13-49ee-983c-f2b730212075 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:26.489531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9a239-8b13-49ee-983c-f2b730212075 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:26.489801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9a239-8b13-49ee-983c-f2b730212075 None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:26.492004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9a239-8b13-49ee-983c-f2b730212075 None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:26.494111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9a239-8b13-49ee-983c-f2b730212075 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:26.496925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 215/430] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:26 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:26.497900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9a239-8b13-49ee-983c-f2b730212075 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:26.504794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 216/431] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:26.506493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d001e4-07cf-4f3a-ac36-9a639d14530e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:26.606756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:26.633745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9a239-8b13-49ee-983c-f2b730212075 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:26.674082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bde95b-e6dc-4600-b4f4-6967978c6bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:26.674717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bde95b-e6dc-4600-b4f4-6967978c6bdf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:26.675437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bde95b-e6dc-4600-b4f4-6967978c6bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:26.675739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bde95b-e6dc-4600-b4f4-6967978c6bdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:26.676479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bde95b-e6dc-4600-b4f4-6967978c6bdf None admin] RBAC: Adding query filter params (group.id=4129d7a0f3ae4e28940f41fd1c1a8776, role.id=1e04fdc0cc334fd7a733a1490a194e37, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:26.676955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:26.677578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bde95b-e6dc-4600-b4f4-6967978c6bdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:26.679172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bde95b-e6dc-4600-b4f4-6967978c6bdf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:26.686538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 216/432] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 05:24:26 2022] GET /identity/v3/role_assignments?group.id=4129d7a0f3ae4e28940f41fd1c1a8776&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:26.730337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d001e4-07cf-4f3a-ac36-9a639d14530e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:26.787576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aacce8-088c-4f9e-a879-c0bf855bda7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:26.788206 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aacce8-088c-4f9e-a879-c0bf855bda7d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:26.788487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aacce8-088c-4f9e-a879-c0bf855bda7d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:26.788762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aacce8-088c-4f9e-a879-c0bf855bda7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:26.789499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aacce8-088c-4f9e-a879-c0bf855bda7d None admin] RBAC: Adding query filter params (group.id=4129d7a0f3ae4e28940f41fd1c1a8776, role.id=cae161e9c1b64656924448dd784ed4d1, scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:26.789806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aacce8-088c-4f9e-a879-c0bf855bda7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:26.791278 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aacce8-088c-4f9e-a879-c0bf855bda7d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:26.803777 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 217/433] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 05:24:26 2022] GET /identity/v3/role_assignments?group.id=4129d7a0f3ae4e28940f41fd1c1a8776&scope.project.id=a0d3c9e097bf4d3ea90c631eadb9d0d9&role.id=cae161e9c1b64656924448dd784ed4d1 => generated 611 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:26.945506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bde95b-e6dc-4600-b4f4-6967978c6bdf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:26.951674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbeb10-d302-4da1-90dc-28a53b78da6d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:26.951674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbeb10-d302-4da1-90dc-28a53b78da6d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:26.951674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbeb10-d302-4da1-90dc-28a53b78da6d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:26.951674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 218/434] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:26.954981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31779e-c4d6-4369-94b8-74388f0a58ca None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:26.957076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31779e-c4d6-4369-94b8-74388f0a58ca None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:26.957576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31779e-c4d6-4369-94b8-74388f0a58ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:26.989594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ed31779e-c4d6-4369-94b8-74388f0a58ca None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:26.991012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed31779e-c4d6-4369-94b8-74388f0a58ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:27.081663 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 219/435] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:27.088646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bde95b-e6dc-4600-b4f4-6967978c6bdf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:27.105680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a993e3-41e1-4711-abdf-ffa873df88f0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:27.106260 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a993e3-41e1-4711-abdf-ffa873df88f0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:27.106260 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a993e3-41e1-4711-abdf-ffa873df88f0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:27.107725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 220/436] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:24:27.113226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771aa86-3c2b-4a3f-a020-236c5e62379a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:27.113680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771aa86-3c2b-4a3f-a020-236c5e62379a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:27.113957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771aa86-3c2b-4a3f-a020-236c5e62379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:27.138352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c771aa86-3c2b-4a3f-a020-236c5e62379a None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:27.139915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c771aa86-3c2b-4a3f-a020-236c5e62379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:27.142662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aee778-eb74-4e9d-8c5a-cb5b86c17804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:27.143521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aee778-eb74-4e9d-8c5a-cb5b86c17804 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:27.143802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aee778-eb74-4e9d-8c5a-cb5b86c17804 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:27.144074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aee778-eb74-4e9d-8c5a-cb5b86c17804 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:27.164502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:27.166273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aee778-eb74-4e9d-8c5a-cb5b86c17804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-16aee778-eb74-4e9d-8c5a-cb5b86c17804 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:27.171755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:27.172748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 05:24:27.173637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:27.180156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 217/437] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:27.182558 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aee778-eb74-4e9d-8c5a-cb5b86c17804 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:27.253284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 221/438] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:27.317691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aee778-eb74-4e9d-8c5a-cb5b86c17804 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:27.360810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8554b09-b4a1-43c4-933a-6a3cd738562c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:27.361822 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8554b09-b4a1-43c4-933a-6a3cd738562c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:27.362328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8554b09-b4a1-43c4-933a-6a3cd738562c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:27.362806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8554b09-b4a1-43c4-933a-6a3cd738562c None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:27.363664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8554b09-b4a1-43c4-933a-6a3cd738562c None admin] RBAC: Adding query filter params (name=member) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:27.364184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8554b09-b4a1-43c4-933a-6a3cd738562c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:27.365791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8554b09-b4a1-43c4-933a-6a3cd738562c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:27.374696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 218/439] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:24:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:27.384351 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771aa86-3c2b-4a3f-a020-236c5e62379a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:27.386465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8554b09-b4a1-43c4-933a-6a3cd738562c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:27.391104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:27.488609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:27.540066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771aa86-3c2b-4a3f-a020-236c5e62379a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:27.542637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8554b09-b4a1-43c4-933a-6a3cd738562c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:27.588601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7bd06-00cf-47f0-bd48-92d54b184d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:27.589478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7bd06-00cf-47f0-bd48-92d54b184d05 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:27.589978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7bd06-00cf-47f0-bd48-92d54b184d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:27.590443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7bd06-00cf-47f0-bd48-92d54b184d05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:27.591460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01b1fb-1134-4b20-883e-6ec380aab837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:27.592070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01b1fb-1134-4b20-883e-6ec380aab837 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:27.592550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01b1fb-1134-4b20-883e-6ec380aab837 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:27.593428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01b1fb-1134-4b20-883e-6ec380aab837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:27.605573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7bd06-00cf-47f0-bd48-92d54b184d05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ddb7bd06-00cf-47f0-bd48-92d54b184d05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:27.613036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:27.614030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 219/440] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:24:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:27.616183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7bd06-00cf-47f0-bd48-92d54b184d05 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:27.679327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01b1fb-1134-4b20-883e-6ec380aab837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:27.680765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01b1fb-1134-4b20-883e-6ec380aab837 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:27.683372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:27.720463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7bd06-00cf-47f0-bd48-92d54b184d05 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:27.742100 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:27.753689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fae14c-84a8-46c2-a4b3-1ccfc5770670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:27.754354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fae14c-84a8-46c2-a4b3-1ccfc5770670 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:27.754794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fae14c-84a8-46c2-a4b3-1ccfc5770670 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:27.755059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fae14c-84a8-46c2-a4b3-1ccfc5770670 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:27.765397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fae14c-84a8-46c2-a4b3-1ccfc5770670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f6fae14c-84a8-46c2-a4b3-1ccfc5770670 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:27.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:27.771077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 220/441] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:24:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:27.772454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fae14c-84a8-46c2-a4b3-1ccfc5770670 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:27.781947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 222/442] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:24:27.782776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01b1fb-1134-4b20-883e-6ec380aab837 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:27.875170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fae14c-84a8-46c2-a4b3-1ccfc5770670 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:27.879726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01b1fb-1134-4b20-883e-6ec380aab837 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:27.882063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:27.912363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c7142-f331-4d04-a906-c96530a2fcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:27.912552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5889e-ebc6-472b-a328-528d4ecba3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:27.912968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c7142-f331-4d04-a906-c96530a2fcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:27.913091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5889e-ebc6-472b-a328-528d4ecba3ed None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:27.913242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c7142-f331-4d04-a906-c96530a2fcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:27.913365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5889e-ebc6-472b-a328-528d4ecba3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:27.913508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c7142-f331-4d04-a906-c96530a2fcb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:27.913632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5889e-ebc6-472b-a328-528d4ecba3ed None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:27.914185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c7142-f331-4d04-a906-c96530a2fcb7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:27.914313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5889e-ebc6-472b-a328-528d4ecba3ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:27.914488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c7142-f331-4d04-a906-c96530a2fcb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:27.914612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5889e-ebc6-472b-a328-528d4ecba3ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:27.915855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c7142-f331-4d04-a906-c96530a2fcb7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:27.916004 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5889e-ebc6-472b-a328-528d4ecba3ed None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:27.924426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 221/443] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:27.926481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 223/444] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:27.926613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c7142-f331-4d04-a906-c96530a2fcb7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:27.930162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5889e-ebc6-472b-a328-528d4ecba3ed None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:27.973164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:28.025864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c7142-f331-4d04-a906-c96530a2fcb7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.026698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5889e-ebc6-472b-a328-528d4ecba3ed None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.059581 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2284ad2-7145-4b26-8e44-327d27e59855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.059775 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d5c64-d5d0-45f3-bd92-4a26adbc6040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.060209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2284ad2-7145-4b26-8e44-327d27e59855 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.060396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5c64-d5d0-45f3-bd92-4a26adbc6040 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.060484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2284ad2-7145-4b26-8e44-327d27e59855 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.060675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5c64-d5d0-45f3-bd92-4a26adbc6040 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.060773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2284ad2-7145-4b26-8e44-327d27e59855 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.060950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5c64-d5d0-45f3-bd92-4a26adbc6040 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.062044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d5c64-d5d0-45f3-bd92-4a26adbc6040 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.063499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d5c64-d5d0-45f3-bd92-4a26adbc6040 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.065105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 222/445] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:27 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:28.065855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d5c64-d5d0-45f3-bd92-4a26adbc6040 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:28.094814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:28.128108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2284ad2-7145-4b26-8e44-327d27e59855 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.129838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2284ad2-7145-4b26-8e44-327d27e59855 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.161731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d5c64-d5d0-45f3-bd92-4a26adbc6040 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.173826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:28.194088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab900c7c-19d1-4072-a8d6-f9d2fcc42c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.194724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab900c7c-19d1-4072-a8d6-f9d2fcc42c98 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.195002 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab900c7c-19d1-4072-a8d6-f9d2fcc42c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.195270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab900c7c-19d1-4072-a8d6-f9d2fcc42c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.226893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 224/446] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:24:28.227735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2284ad2-7145-4b26-8e44-327d27e59855 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:28.265489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab900c7c-19d1-4072-a8d6-f9d2fcc42c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.266970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab900c7c-19d1-4072-a8d6-f9d2fcc42c98 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.289603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:28.323772 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2284ad2-7145-4b26-8e44-327d27e59855 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.358501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d8e8a-81ca-4053-b5ad-7d387a6ecc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.359122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d8e8a-81ca-4053-b5ad-7d387a6ecc1d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.359398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d8e8a-81ca-4053-b5ad-7d387a6ecc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.359666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d8e8a-81ca-4053-b5ad-7d387a6ecc1d None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.360763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d8e8a-81ca-4053-b5ad-7d387a6ecc1d None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.362753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d8e8a-81ca-4053-b5ad-7d387a6ecc1d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.363738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 223/447] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:28.364427 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 225/448] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:28 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:28.364567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab900c7c-19d1-4072-a8d6-f9d2fcc42c98 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:28.369235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d8e8a-81ca-4053-b5ad-7d387a6ecc1d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:28.378109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:28.460319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab900c7c-19d1-4072-a8d6-f9d2fcc42c98 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.467011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d8e8a-81ca-4053-b5ad-7d387a6ecc1d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.493803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bac1a0-166a-47a7-ba7e-43e76ff07a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.494643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac1a0-166a-47a7-ba7e-43e76ff07a2a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.494914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac1a0-166a-47a7-ba7e-43e76ff07a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.495181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac1a0-166a-47a7-ba7e-43e76ff07a2a None admin] PATH_INFO: `/v3/projects/4c2b6383659b48fcabc56888b6bfb190` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.496280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bac1a0-166a-47a7-ba7e-43e76ff07a2a None admin] RBAC: Authorizing `identity:get_project(project_id=4c2b6383659b48fcabc56888b6bfb190)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.497694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bac1a0-166a-47a7-ba7e-43e76ff07a2a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.499095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c611095-154f-4b92-a8ff-49dc182869e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.499325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 224/449] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:28 2022] GET /identity/v3/projects/4c2b6383659b48fcabc56888b6bfb190 => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:28.499719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c611095-154f-4b92-a8ff-49dc182869e5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.499998 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c611095-154f-4b92-a8ff-49dc182869e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.500088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bac1a0-166a-47a7-ba7e-43e76ff07a2a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:28.500270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c611095-154f-4b92-a8ff-49dc182869e5 None admin] PATH_INFO: `/v3/projects/ce294031877d49d18dd64a30dbcc7c63` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.504195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c611095-154f-4b92-a8ff-49dc182869e5 None admin] RBAC: Authorizing `identity:get_project(project_id=ce294031877d49d18dd64a30dbcc7c63)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.505596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c611095-154f-4b92-a8ff-49dc182869e5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.507218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 226/450] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:24:28 2022] GET /identity/v3/projects/ce294031877d49d18dd64a30dbcc7c63 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:28.507953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c611095-154f-4b92-a8ff-49dc182869e5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:28.510484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:28.581686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:28.599926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bac1a0-166a-47a7-ba7e-43e76ff07a2a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.605922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c611095-154f-4b92-a8ff-49dc182869e5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.632571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338a90f-d6aa-4a03-8913-e4d53050850e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.633165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338a90f-d6aa-4a03-8913-e4d53050850e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.633437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338a90f-d6aa-4a03-8913-e4d53050850e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.633704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338a90f-d6aa-4a03-8913-e4d53050850e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.634436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338a90f-d6aa-4a03-8913-e4d53050850e None admin] RBAC: Adding query filter params (group.id=4129d7a0f3ae4e28940f41fd1c1a8776, role.id=1e04fdc0cc334fd7a733a1490a194e37, scope.project.id=ce294031877d49d18dd64a30dbcc7c63) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:28.634741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338a90f-d6aa-4a03-8913-e4d53050850e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.636137 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338a90f-d6aa-4a03-8913-e4d53050850e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.637890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1e146-fc4b-4812-9a90-5dffde957002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.638539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1e146-fc4b-4812-9a90-5dffde957002 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.638812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1e146-fc4b-4812-9a90-5dffde957002 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.639091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1e146-fc4b-4812-9a90-5dffde957002 None admin] PATH_INFO: `/v3/groups/322918b086f34f49a61bb04617ebcdc9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.640181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1e146-fc4b-4812-9a90-5dffde957002 None admin] RBAC: Authorizing `identity:get_group(group_id=322918b086f34f49a61bb04617ebcdc9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.641570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1e146-fc4b-4812-9a90-5dffde957002 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.642253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 225/451] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 05:24:28 2022] GET /identity/v3/role_assignments?group.id=4129d7a0f3ae4e28940f41fd1c1a8776&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=1e04fdc0cc334fd7a733a1490a194e37 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:28.643919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338a90f-d6aa-4a03-8913-e4d53050850e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:28.644023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 227/452] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:28 2022] GET /identity/v3/groups/322918b086f34f49a61bb04617ebcdc9 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:28.655457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1e146-fc4b-4812-9a90-5dffde957002 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:28.735309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:28.759082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338a90f-d6aa-4a03-8913-e4d53050850e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.768644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1e146-fc4b-4812-9a90-5dffde957002 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.796049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee721-b4af-4589-911a-2a85cc21e486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.796699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee721-b4af-4589-911a-2a85cc21e486 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.796974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee721-b4af-4589-911a-2a85cc21e486 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.797253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee721-b4af-4589-911a-2a85cc21e486 None admin] PATH_INFO: `/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.798437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ee721-b4af-4589-911a-2a85cc21e486 None admin] RBAC: Authorizing `identity:get_group(group_id=4129d7a0f3ae4e28940f41fd1c1a8776)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.799954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ee721-b4af-4589-911a-2a85cc21e486 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.806525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 226/453] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 05:24:28 2022] GET /identity/v3/groups/4129d7a0f3ae4e28940f41fd1c1a8776 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:28.807841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee721-b4af-4589-911a-2a85cc21e486 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:28.810045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf04d1-03da-4209-a53e-8adea16b538e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.810688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf04d1-03da-4209-a53e-8adea16b538e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.810966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf04d1-03da-4209-a53e-8adea16b538e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.811239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf04d1-03da-4209-a53e-8adea16b538e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.811972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf04d1-03da-4209-a53e-8adea16b538e None admin] RBAC: Adding query filter params (group.id=322918b086f34f49a61bb04617ebcdc9, role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=4c2b6383659b48fcabc56888b6bfb190) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:28.812361 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf04d1-03da-4209-a53e-8adea16b538e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.813684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf04d1-03da-4209-a53e-8adea16b538e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.822518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 228/454] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 05:24:28 2022] GET /identity/v3/role_assignments?group.id=322918b086f34f49a61bb04617ebcdc9&scope.project.id=4c2b6383659b48fcabc56888b6bfb190&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:28.824755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:28.931153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee721-b4af-4589-911a-2a85cc21e486 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:28.972754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe23835-a544-4657-8728-03ad1426aa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:28.973383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe23835-a544-4657-8728-03ad1426aa2b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:28.973659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe23835-a544-4657-8728-03ad1426aa2b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:28.973926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe23835-a544-4657-8728-03ad1426aa2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:28.974718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe23835-a544-4657-8728-03ad1426aa2b None admin] RBAC: Adding query filter params (group.id=4129d7a0f3ae4e28940f41fd1c1a8776, role.id=cae161e9c1b64656924448dd784ed4d1, scope.project.id=ce294031877d49d18dd64a30dbcc7c63) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:28.975030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe23835-a544-4657-8728-03ad1426aa2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:28.976617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe23835-a544-4657-8728-03ad1426aa2b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:28.983508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 227/455] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 05:24:28 2022] GET /identity/v3/role_assignments?group.id=4129d7a0f3ae4e28940f41fd1c1a8776&scope.project.id=ce294031877d49d18dd64a30dbcc7c63&role.id=cae161e9c1b64656924448dd784ed4d1 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:31.860368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4042da-9725-4eca-8236-66e042633134 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:31.860882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4042da-9725-4eca-8236-66e042633134 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:31.860957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4042da-9725-4eca-8236-66e042633134 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:31.862572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 229/456] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:31.867944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfce0b4-fbdb-404f-9e12-7ec71939fa7f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:31.868368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfce0b4-fbdb-404f-9e12-7ec71939fa7f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:31.868543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfce0b4-fbdb-404f-9e12-7ec71939fa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:31.894972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-edfce0b4-fbdb-404f-9e12-7ec71939fa7f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:31.896161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-edfce0b4-fbdb-404f-9e12-7ec71939fa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:31.964203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7c39b-669f-4fdb-917e-8539e056344a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:31.964516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7c39b-669f-4fdb-917e-8539e056344a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:31.964754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7c39b-669f-4fdb-917e-8539e056344a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:31.966323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 230/457] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:31.971371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3cf5d-4557-4dab-987a-c70c77dcd9e0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:31.971653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3cf5d-4557-4dab-987a-c70c77dcd9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:31.971926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3cf5d-4557-4dab-987a-c70c77dcd9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:31.988545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 228/458] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:31.999535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d7a3cf5d-4557-4dab-987a-c70c77dcd9e0 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:32.000716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7a3cf5d-4557-4dab-987a-c70c77dcd9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:32.101931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 231/459] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:32.253330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a9a63-3de5-44a0-b37c-9fbe595844ab None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:32.253623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a9a63-3de5-44a0-b37c-9fbe595844ab None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:32.253806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a9a63-3de5-44a0-b37c-9fbe595844ab None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:32.255178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 229/460] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:32.257104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3cf5d-4557-4dab-987a-c70c77dcd9e0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:32.262046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bec57-0545-46ea-98a3-1f490183db40 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:32.262342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bec57-0545-46ea-98a3-1f490183db40 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:32.262570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bec57-0545-46ea-98a3-1f490183db40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:32.281727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:32.291604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-210bec57-0545-46ea-98a3-1f490183db40 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:32.292843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-210bec57-0545-46ea-98a3-1f490183db40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:32.338498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:32.412546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3cf5d-4557-4dab-987a-c70c77dcd9e0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:32.427355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 230/461] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:32.462673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bec57-0545-46ea-98a3-1f490183db40 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:32.470318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f7d9c-e8ab-4f6d-812f-5c83f48ac5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:32.471290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f7d9c-e8ab-4f6d-812f-5c83f48ac5ea None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:32.471715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f7d9c-e8ab-4f6d-812f-5c83f48ac5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:32.472129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f7d9c-e8ab-4f6d-812f-5c83f48ac5ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:32.483241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f7d9c-e8ab-4f6d-812f-5c83f48ac5ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:32.485023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f7d9c-e8ab-4f6d-812f-5c83f48ac5ea None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-505f7d9c-e8ab-4f6d-812f-5c83f48ac5ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:32.503192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:32.504366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:32.505408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:32.505408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:32.505408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:32.512379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 232/462] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:32.512379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1bad9-51ea-4a3b-b803-105ec9e01c3b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:32.512379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1bad9-51ea-4a3b-b803-105ec9e01c3b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:32.512379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1bad9-51ea-4a3b-b803-105ec9e01c3b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:32.514052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 233/463] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:32.514613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1bad9-51ea-4a3b-b803-105ec9e01c3b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:32.591382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bec57-0545-46ea-98a3-1f490183db40 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:32.628367 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc5dee-7c35-4e07-9e2a-16ecdee26f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:32.628999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5dee-7c35-4e07-9e2a-16ecdee26f91 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:32.629276 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5dee-7c35-4e07-9e2a-16ecdee26f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:32.629545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5dee-7c35-4e07-9e2a-16ecdee26f91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:32.632505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:32.640204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1bad9-51ea-4a3b-b803-105ec9e01c3b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:32.640825 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc5dee-7c35-4e07-9e2a-16ecdee26f91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:32.642624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc5dee-7c35-4e07-9e2a-16ecdee26f91 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:32.656958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-82bc5dee-7c35-4e07-9e2a-16ecdee26f91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:32.659902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:32.660781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:32.661601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:32.661601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:32.661601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:32.666162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 231/464] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:32.667288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d2bc-57ec-4687-99dc-42acb113e252 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:32.667568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d2bc-57ec-4687-99dc-42acb113e252 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:32.667837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d2bc-57ec-4687-99dc-42acb113e252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:32.677447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb112b0-98a0-4226-9ab4-2faf3b4cf8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:32.678058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb112b0-98a0-4226-9ab4-2faf3b4cf8aa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:32.678368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb112b0-98a0-4226-9ab4-2faf3b4cf8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:32.678646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb112b0-98a0-4226-9ab4-2faf3b4cf8aa None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:32.679410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb112b0-98a0-4226-9ab4-2faf3b4cf8aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:32.679719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb112b0-98a0-4226-9ab4-2faf3b4cf8aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:32.681097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb112b0-98a0-4226-9ab4-2faf3b4cf8aa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:32.687509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d441d2bc-57ec-4687-99dc-42acb113e252 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:32.688704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d441d2bc-57ec-4687-99dc-42acb113e252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:32.693056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 234/465] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:32.693935 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb112b0-98a0-4226-9ab4-2faf3b4cf8aa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:32.762854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 232/466] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:32.763397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441d2bc-57ec-4687-99dc-42acb113e252 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:32.806725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb112b0-98a0-4226-9ab4-2faf3b4cf8aa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:32.843179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b1f22-d20d-4f29-b4c7-345ccf2e26ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:32.843790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b1f22-d20d-4f29-b4c7-345ccf2e26ee None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:32.844065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b1f22-d20d-4f29-b4c7-345ccf2e26ee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:32.844336 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b1f22-d20d-4f29-b4c7-345ccf2e26ee None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:32.844971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b1f22-d20d-4f29-b4c7-345ccf2e26ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:32.845269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b1f22-d20d-4f29-b4c7-345ccf2e26ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:32.847628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b1f22-d20d-4f29-b4c7-345ccf2e26ee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:32.860224 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 235/467] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:32.865531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b1f22-d20d-4f29-b4c7-345ccf2e26ee None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:32.874993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441d2bc-57ec-4687-99dc-42acb113e252 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:32.899665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:32.914121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:32.921445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e572d40-e71b-438f-a31a-a99bd2b1968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:32.922362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e572d40-e71b-438f-a31a-a99bd2b1968c None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:32.923088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e572d40-e71b-438f-a31a-a99bd2b1968c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:32.923826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e572d40-e71b-438f-a31a-a99bd2b1968c None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:32.924660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e572d40-e71b-438f-a31a-a99bd2b1968c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:32.926958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e572d40-e71b-438f-a31a-a99bd2b1968c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:32.978905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b1f22-d20d-4f29-b4c7-345ccf2e26ee None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:33.010910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a5119-d9ef-42c9-b940-f50eb7aed80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:33.011532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a5119-d9ef-42c9-b940-f50eb7aed80d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:33.011806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a5119-d9ef-42c9-b940-f50eb7aed80d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:33.012073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a5119-d9ef-42c9-b940-f50eb7aed80d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:33.022667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a5119-d9ef-42c9-b940-f50eb7aed80d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:33.024057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a5119-d9ef-42c9-b940-f50eb7aed80d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c42a5119-d9ef-42c9-b940-f50eb7aed80d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:33.039060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:33.040069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:33.047930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:33.047930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:33.047930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:33.047930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 236/468] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:33.047930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a5119-d9ef-42c9-b940-f50eb7aed80d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:33.049365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 233/469] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:24:32 2022] POST /identity/v3/users => generated 248 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:24:33.051633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e572d40-e71b-438f-a31a-a99bd2b1968c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:33.163953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:33.188575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a5119-d9ef-42c9-b940-f50eb7aed80d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:33.200647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e572d40-e71b-438f-a31a-a99bd2b1968c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:33.239719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d25c1-fdf3-4835-bc2c-2e1d11b3394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:33.239919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98b91b-f2fa-4b73-b780-ed0f48155149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:33.240169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d25c1-fdf3-4835-bc2c-2e1d11b3394c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:33.240344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98b91b-f2fa-4b73-b780-ed0f48155149 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:33.240436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d25c1-fdf3-4835-bc2c-2e1d11b3394c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:33.240579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98b91b-f2fa-4b73-b780-ed0f48155149 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:33.240667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d25c1-fdf3-4835-bc2c-2e1d11b3394c None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:33.240834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98b91b-f2fa-4b73-b780-ed0f48155149 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:33.241280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d25c1-fdf3-4835-bc2c-2e1d11b3394c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:33.241475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98b91b-f2fa-4b73-b780-ed0f48155149 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:33.241567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d25c1-fdf3-4835-bc2c-2e1d11b3394c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:33.244001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d25c1-fdf3-4835-bc2c-2e1d11b3394c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:33.244107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98b91b-f2fa-4b73-b780-ed0f48155149 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:33.265754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 234/470] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:33.266732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d25c1-fdf3-4835-bc2c-2e1d11b3394c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:33.304820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:33.335396 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 237/471] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:24:33 2022] POST /identity/v3/users => generated 246 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:24:33.336284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98b91b-f2fa-4b73-b780-ed0f48155149 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:33.375993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d25c1-fdf3-4835-bc2c-2e1d11b3394c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:33.416471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96738caf-2783-47b5-bf1e-2e945bafbb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:33.417444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96738caf-2783-47b5-bf1e-2e945bafbb2d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:33.417790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96738caf-2783-47b5-bf1e-2e945bafbb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:33.418161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96738caf-2783-47b5-bf1e-2e945bafbb2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:33.429604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96738caf-2783-47b5-bf1e-2e945bafbb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:33.431212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96738caf-2783-47b5-bf1e-2e945bafbb2d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-96738caf-2783-47b5-bf1e-2e945bafbb2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:33.448234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:33.449397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:33.450482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:33.450482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:33.450482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:33.451498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 235/472] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:33.452905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca2cec-38d7-4b34-a01f-cd0e5f4c944e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:33.453256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca2cec-38d7-4b34-a01f-cd0e5f4c944e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:33.453524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca2cec-38d7-4b34-a01f-cd0e5f4c944e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:33.455692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 236/473] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:33.457577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca2cec-38d7-4b34-a01f-cd0e5f4c944e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:33.478796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98b91b-f2fa-4b73-b780-ed0f48155149 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:33.521903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26922b0-3db9-4e50-aa18-3e16ae44b5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:33.522729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26922b0-3db9-4e50-aa18-3e16ae44b5c1 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:33.523065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26922b0-3db9-4e50-aa18-3e16ae44b5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:33.523402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26922b0-3db9-4e50-aa18-3e16ae44b5c1 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:33.524132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26922b0-3db9-4e50-aa18-3e16ae44b5c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:33.526635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26922b0-3db9-4e50-aa18-3e16ae44b5c1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:33.562394 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:33.581445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca2cec-38d7-4b34-a01f-cd0e5f4c944e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:33.619309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a54b5e-6773-430a-94d5-41897213bc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:33.619886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a54b5e-6773-430a-94d5-41897213bc0f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:33.620196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a54b5e-6773-430a-94d5-41897213bc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:33.620467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a54b5e-6773-430a-94d5-41897213bc0f None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:33.621104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a54b5e-6773-430a-94d5-41897213bc0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:33.621408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a54b5e-6773-430a-94d5-41897213bc0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:33.622845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a54b5e-6773-430a-94d5-41897213bc0f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:33.630965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:33.635616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 238/474] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:24:33 2022] POST /identity/v3/users => generated 248 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:24:33.636305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 237/475] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:33.636848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be88e2-fea6-4aa7-b230-da167f3c1292 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:33.637179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be88e2-fea6-4aa7-b230-da167f3c1292 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:33.642987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a54b5e-6773-430a-94d5-41897213bc0f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:33.644517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be88e2-fea6-4aa7-b230-da167f3c1292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:33.676669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-01be88e2-fea6-4aa7-b230-da167f3c1292 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:33.681643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-01be88e2-fea6-4aa7-b230-da167f3c1292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:33.819478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 239/476] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:33.854509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a54b5e-6773-430a-94d5-41897213bc0f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:33.899719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4630806-007a-4fe5-ad22-8f899c31e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:33.901050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630806-007a-4fe5-ad22-8f899c31e4e4 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:33.901553 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630806-007a-4fe5-ad22-8f899c31e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:33.902049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630806-007a-4fe5-ad22-8f899c31e4e4 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:33.903055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4630806-007a-4fe5-ad22-8f899c31e4e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:33.904671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4630806-007a-4fe5-ad22-8f899c31e4e4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:34.039195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 238/477] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:24:33 2022] POST /identity/v3/users => generated 249 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:24:34.108619 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be88e2-fea6-4aa7-b230-da167f3c1292 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:34.254775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be88e2-fea6-4aa7-b230-da167f3c1292 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:34.289410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:34.292589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d27f42-1449-426f-a24d-ac7abfa67e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:34.293203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d27f42-1449-426f-a24d-ac7abfa67e8a None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:34.293483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d27f42-1449-426f-a24d-ac7abfa67e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:34.293754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d27f42-1449-426f-a24d-ac7abfa67e8a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:34.294452 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d27f42-1449-426f-a24d-ac7abfa67e8a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:34.295816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d27f42-1449-426f-a24d-ac7abfa67e8a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:34.343631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.sql.core [None req-c1d27f42-1449-426f-a24d-ac7abfa67e8a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 05:24:34.343631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 05:24:34.343631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [parameters: {'id': '7ce100b87e2e466585a49a3e6e79bb35', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 05:24:34.343631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81480) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c1d27f42-1449-426f-a24d-ac7abfa67e8a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Oct 04 05:24:34.361834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return ref.to_dict() Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 05:24:34.362792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application yield resource Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise self.value Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application session.commit() Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 05:24:34.363724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.flush() Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application compat.raise_( Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 05:24:34.364572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 05:24:34.365520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 05:24:34.366395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [parameters: {'id': '7ce100b87e2e466585a49a3e6e79bb35', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:34.367187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 05:24:34.367972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:34.368826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 240/478] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Oct 4 05:24:34 2022] POST /identity/v3/roles => generated 144 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:34.377077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4630806-007a-4fe5-ad22-8f899c31e4e4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:34.380396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:34.512749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4630806-007a-4fe5-ad22-8f899c31e4e4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:34.566833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04b520-8312-4e96-afe7-b08635b4f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:34.567474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b520-8312-4e96-afe7-b08635b4f601 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:34.567752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b520-8312-4e96-afe7-b08635b4f601 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:34.568025 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b520-8312-4e96-afe7-b08635b4f601 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:34.578731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04b520-8312-4e96-afe7-b08635b4f601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-aa04b520-8312-4e96-afe7-b08635b4f601 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:34.584488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:34.585392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:34.586225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 239/479] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:24:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:34.593093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d27f42-1449-426f-a24d-ac7abfa67e8a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:34.737245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d27f42-1449-426f-a24d-ac7abfa67e8a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:34.778975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2793763-f47b-469f-92b0-1235a9b5634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:34.779583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2793763-f47b-469f-92b0-1235a9b5634a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:34.779855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2793763-f47b-469f-92b0-1235a9b5634a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:34.780123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2793763-f47b-469f-92b0-1235a9b5634a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:34.780785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2793763-f47b-469f-92b0-1235a9b5634a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:34.781084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2793763-f47b-469f-92b0-1235a9b5634a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:34.782469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2793763-f47b-469f-92b0-1235a9b5634a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:34.793322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 241/480] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:35.471640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1bb32-af12-472b-9447-df5a07a11c90 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:35.472155 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1bb32-af12-472b-9447-df5a07a11c90 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:35.472155 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1bb32-af12-472b-9447-df5a07a11c90 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:35.473704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 240/481] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:35.483615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201e1a4-49da-4989-b27f-ab5af1ab2d15 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:35.484018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201e1a4-49da-4989-b27f-ab5af1ab2d15 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:35.484209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201e1a4-49da-4989-b27f-ab5af1ab2d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:35.503346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2201e1a4-49da-4989-b27f-ab5af1ab2d15 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:35.504515 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2201e1a4-49da-4989-b27f-ab5af1ab2d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:35.565532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:35.587616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 242/482] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:35.925505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30844a-89b6-49e1-b112-1b7dd49e0fc7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:35.926117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30844a-89b6-49e1-b112-1b7dd49e0fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:35.926600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30844a-89b6-49e1-b112-1b7dd49e0fc7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:35.928169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 241/483] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:35.933328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fb3c0-0be5-4e4f-9b75-7cf3b59278a2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:35.933835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fb3c0-0be5-4e4f-9b75-7cf3b59278a2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:35.934329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8fb3c0-0be5-4e4f-9b75-7cf3b59278a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:35.959078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3a8fb3c0-0be5-4e4f-9b75-7cf3b59278a2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:35.960569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3a8fb3c0-0be5-4e4f-9b75-7cf3b59278a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:36.005507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30844a-89b6-49e1-b112-1b7dd49e0fc7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:36.059727 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 243/484] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:36.104545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:36.160265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30844a-89b6-49e1-b112-1b7dd49e0fc7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:36.215792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddc905-06c7-4437-8452-43516dcf6756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:36.216438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddc905-06c7-4437-8452-43516dcf6756 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:36.216718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddc905-06c7-4437-8452-43516dcf6756 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:36.216987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddc905-06c7-4437-8452-43516dcf6756 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:36.231976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eddc905-06c7-4437-8452-43516dcf6756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6eddc905-06c7-4437-8452-43516dcf6756 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:36.238016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:36.238903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:36.239697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:36.243867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 242/485] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:36.244843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fb3c0-0be5-4e4f-9b75-7cf3b59278a2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:36.295188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54632a0c-6023-4303-985e-2c415c94a744 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:36.295522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54632a0c-6023-4303-985e-2c415c94a744 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:36.295730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54632a0c-6023-4303-985e-2c415c94a744 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:36.297197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 243/486] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:36.302734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999ea53-9285-4022-a3dd-056b0255d293 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:36.303013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999ea53-9285-4022-a3dd-056b0255d293 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:36.303282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999ea53-9285-4022-a3dd-056b0255d293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:36.332872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0999ea53-9285-4022-a3dd-056b0255d293 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:36.334081 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0999ea53-9285-4022-a3dd-056b0255d293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:36.392996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8fb3c0-0be5-4e4f-9b75-7cf3b59278a2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:36.408501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 244/487] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:36.411588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999ea53-9285-4022-a3dd-056b0255d293 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:36.413474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:36.464109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fb03e-de29-4309-88b5-5003e17390ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:36.465885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fb03e-de29-4309-88b5-5003e17390ac None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:36.466492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fb03e-de29-4309-88b5-5003e17390ac None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:36.467647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fb03e-de29-4309-88b5-5003e17390ac None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:36.468784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fb03e-de29-4309-88b5-5003e17390ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:36.469446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fb03e-de29-4309-88b5-5003e17390ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:36.472325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fb03e-de29-4309-88b5-5003e17390ac None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:36.494078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 244/488] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:36.495631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fb03e-de29-4309-88b5-5003e17390ac None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:36.527151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:36.579662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999ea53-9285-4022-a3dd-056b0255d293 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:36.631216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d95ab-9e48-4c42-931b-44f2c723ff34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:36.634637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d95ab-9e48-4c42-931b-44f2c723ff34 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:36.635257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d95ab-9e48-4c42-931b-44f2c723ff34 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:36.635753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d95ab-9e48-4c42-931b-44f2c723ff34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:36.669201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d95ab-9e48-4c42-931b-44f2c723ff34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f62d95ab-9e48-4c42-931b-44f2c723ff34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:36.674424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:36.675313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:36.676153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 245/489] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:36.691095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d95ab-9e48-4c42-931b-44f2c723ff34 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:36.707932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fb03e-de29-4309-88b5-5003e17390ac None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:36.743273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbc3b3-b8b9-40dd-acec-045baf8d74d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:36.744006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbc3b3-b8b9-40dd-acec-045baf8d74d5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:36.744288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbc3b3-b8b9-40dd-acec-045baf8d74d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:36.744562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbc3b3-b8b9-40dd-acec-045baf8d74d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:36.763034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbc3b3-b8b9-40dd-acec-045baf8d74d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:36.768864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbc3b3-b8b9-40dd-acec-045baf8d74d5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6ebbc3b3-b8b9-40dd-acec-045baf8d74d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:36.786621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:36.787717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:36.788820 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:36.788820 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:36.788820 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:36.788820 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 245/490] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:36.788820 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbc3b3-b8b9-40dd-acec-045baf8d74d5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:36.801856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d95ab-9e48-4c42-931b-44f2c723ff34 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:36.815554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:36.823625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:36.836313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25b888-60d3-4682-9fbe-73a6bdb255a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:36.836915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25b888-60d3-4682-9fbe-73a6bdb255a8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:36.837189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25b888-60d3-4682-9fbe-73a6bdb255a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:36.837462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25b888-60d3-4682-9fbe-73a6bdb255a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:36.838117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25b888-60d3-4682-9fbe-73a6bdb255a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:36.840882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25b888-60d3-4682-9fbe-73a6bdb255a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:36.840882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25b888-60d3-4682-9fbe-73a6bdb255a8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:36.848522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 246/491] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:36.849733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cee1f0-c503-484d-8680-18c56e74805b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:36.850096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cee1f0-c503-484d-8680-18c56e74805b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:36.850467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cee1f0-c503-484d-8680-18c56e74805b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:36.851913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 247/492] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:36.856433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cee1f0-c503-484d-8680-18c56e74805b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:36.910004 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbc3b3-b8b9-40dd-acec-045baf8d74d5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:36.948213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619c4e9-8817-4e02-a7d6-29a5eef2b024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:36.948777 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c4e9-8817-4e02-a7d6-29a5eef2b024 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:36.949058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c4e9-8817-4e02-a7d6-29a5eef2b024 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:36.949331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c4e9-8817-4e02-a7d6-29a5eef2b024 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:36.961015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619c4e9-8817-4e02-a7d6-29a5eef2b024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:36.963256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cee1f0-c503-484d-8680-18c56e74805b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6619c4e9-8817-4e02-a7d6-29a5eef2b024 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:36.965849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:36.966874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:36.967857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 246/493] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:36.968633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619c4e9-8817-4e02-a7d6-29a5eef2b024 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.008912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1060471-448c-447b-b059-12a31a79b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.009910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1060471-448c-447b-b059-12a31a79b8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.010226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1060471-448c-447b-b059-12a31a79b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.010501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1060471-448c-447b-b059-12a31a79b8b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.011144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1060471-448c-447b-b059-12a31a79b8b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:37.011452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1060471-448c-447b-b059-12a31a79b8b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:37.013506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1060471-448c-447b-b059-12a31a79b8b5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:37.025919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 248/494] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:37.027233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f8812-b6a4-482a-98ed-1d7b35680d7e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.027520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f8812-b6a4-482a-98ed-1d7b35680d7e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.027791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f8812-b6a4-482a-98ed-1d7b35680d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.045370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ae0f8812-b6a4-482a-98ed-1d7b35680d7e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:37.046664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ae0f8812-b6a4-482a-98ed-1d7b35680d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:37.070649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:37.077084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:37.081487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619c4e9-8817-4e02-a7d6-29a5eef2b024 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.115872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 249/495] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:37.116460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4464a5-ac46-4a5a-a201-ff278b595244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.116700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f8812-b6a4-482a-98ed-1d7b35680d7e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.117584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4464a5-ac46-4a5a-a201-ff278b595244 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.117990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4464a5-ac46-4a5a-a201-ff278b595244 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.118416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4464a5-ac46-4a5a-a201-ff278b595244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.137030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4464a5-ac46-4a5a-a201-ff278b595244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:37.138527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4464a5-ac46-4a5a-a201-ff278b595244 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-be4464a5-ac46-4a5a-a201-ff278b595244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:37.153948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:37.155082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:37.156142 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:37.156142 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:37.156142 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:37.159769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 247/496] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:37.160587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4464a5-ac46-4a5a-a201-ff278b595244 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.245416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f8812-b6a4-482a-98ed-1d7b35680d7e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.270375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4464a5-ac46-4a5a-a201-ff278b595244 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.290466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedab253-399c-4b62-a42c-1408b7690d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.291188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedab253-399c-4b62-a42c-1408b7690d95 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.291559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedab253-399c-4b62-a42c-1408b7690d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.292014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedab253-399c-4b62-a42c-1408b7690d95 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.292701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedab253-399c-4b62-a42c-1408b7690d95 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:37.293099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedab253-399c-4b62-a42c-1408b7690d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:37.294792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedab253-399c-4b62-a42c-1408b7690d95 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:37.308200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 250/497] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:37.308200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedab253-399c-4b62-a42c-1408b7690d95 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.310499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9934c-d6a0-4ef1-be29-1b14b340718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.311357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9934c-d6a0-4ef1-be29-1b14b340718d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.311826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9934c-d6a0-4ef1-be29-1b14b340718d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.312284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9934c-d6a0-4ef1-be29-1b14b340718d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.321124 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9934c-d6a0-4ef1-be29-1b14b340718d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:37.322861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9934c-d6a0-4ef1-be29-1b14b340718d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:37.334957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d8a9934c-d6a0-4ef1-be29-1b14b340718d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:37.336526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:37.337382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:37.338310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:37.338310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:37.338310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:37.338310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:37.338310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:37.338310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:37.338310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:37.338310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:37.338310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 05:24:37.338310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:37.344966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 248/498] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:37.346051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9934c-d6a0-4ef1-be29-1b14b340718d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.351978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:37.414314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedab253-399c-4b62-a42c-1408b7690d95 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.449196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f40f4-8afb-4c95-9ac3-182087707f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.449812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f40f4-8afb-4c95-9ac3-182087707f76 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.450113 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f40f4-8afb-4c95-9ac3-182087707f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.450379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f40f4-8afb-4c95-9ac3-182087707f76 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.450585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9934c-d6a0-4ef1-be29-1b14b340718d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.451012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f40f4-8afb-4c95-9ac3-182087707f76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:37.451308 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f40f4-8afb-4c95-9ac3-182087707f76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:37.452659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f40f4-8afb-4c95-9ac3-182087707f76 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:37.468307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 251/499] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:37.468307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f40f4-8afb-4c95-9ac3-182087707f76 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.490001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac336b7c-ec85-4d61-8ffe-34c90931d768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.490671 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac336b7c-ec85-4d61-8ffe-34c90931d768 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.490949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac336b7c-ec85-4d61-8ffe-34c90931d768 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.491217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac336b7c-ec85-4d61-8ffe-34c90931d768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.502980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac336b7c-ec85-4d61-8ffe-34c90931d768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:37.504284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac336b7c-ec85-4d61-8ffe-34c90931d768 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ac336b7c-ec85-4d61-8ffe-34c90931d768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:37.521205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:37.522545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:37.523822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:37.523822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:37.523822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:37.523822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 249/500] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:37.523822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac336b7c-ec85-4d61-8ffe-34c90931d768 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.563442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:37.580080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f40f4-8afb-4c95-9ac3-182087707f76 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.581038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:37.615113 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85037a8a-4b93-42e2-9c5c-49b41f92665b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.615720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85037a8a-4b93-42e2-9c5c-49b41f92665b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.615996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85037a8a-4b93-42e2-9c5c-49b41f92665b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.616300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85037a8a-4b93-42e2-9c5c-49b41f92665b None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.617017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85037a8a-4b93-42e2-9c5c-49b41f92665b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:37.617322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85037a8a-4b93-42e2-9c5c-49b41f92665b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:37.618770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85037a8a-4b93-42e2-9c5c-49b41f92665b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:37.626490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac336b7c-ec85-4d61-8ffe-34c90931d768 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.658633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 252/501] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 05:24:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:37.658633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85037a8a-4b93-42e2-9c5c-49b41f92665b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.664962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34dd741-ee93-491e-9e43-4df0bc5d8c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.665587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dd741-ee93-491e-9e43-4df0bc5d8c65 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.665860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dd741-ee93-491e-9e43-4df0bc5d8c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.666127 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34dd741-ee93-491e-9e43-4df0bc5d8c65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.677300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34dd741-ee93-491e-9e43-4df0bc5d8c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b34dd741-ee93-491e-9e43-4df0bc5d8c65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:37.682604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:37.683531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 250/502] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:37.684904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34dd741-ee93-491e-9e43-4df0bc5d8c65 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.774425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85037a8a-4b93-42e2-9c5c-49b41f92665b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.791298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34dd741-ee93-491e-9e43-4df0bc5d8c65 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.806971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:37.812103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b22e2-3a3c-4899-a90a-8bc3f1f0604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.812757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b22e2-3a3c-4899-a90a-8bc3f1f0604d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.813030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b22e2-3a3c-4899-a90a-8bc3f1f0604d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.813295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b22e2-3a3c-4899-a90a-8bc3f1f0604d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.821293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b22e2-3a3c-4899-a90a-8bc3f1f0604d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:37.822816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b22e2-3a3c-4899-a90a-8bc3f1f0604d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:37.828529 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b771c4-0208-418a-b847-729465df3794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.829163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b771c4-0208-418a-b847-729465df3794 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.829542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b771c4-0208-418a-b847-729465df3794 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.829945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b771c4-0208-418a-b847-729465df3794 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:37.830803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b771c4-0208-418a-b847-729465df3794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:37.831240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b771c4-0208-418a-b847-729465df3794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:37.832776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b771c4-0208-418a-b847-729465df3794 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-653b22e2-3a3c-4899-a90a-8bc3f1f0604d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:37.837336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:37.838501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:37.839604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 253/503] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 05:24:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:37.843169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b22e2-3a3c-4899-a90a-8bc3f1f0604d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.847879 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:37.856092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 251/504] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:37.857203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b771c4-0208-418a-b847-729465df3794 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:37.954933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b22e2-3a3c-4899-a90a-8bc3f1f0604d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.978037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b771c4-0208-418a-b847-729465df3794 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:37.990608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e297b-4c92-4d05-81c4-50f3c881816b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:37.991216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e297b-4c92-4d05-81c4-50f3c881816b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:37.991490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e297b-4c92-4d05-81c4-50f3c881816b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:37.991759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e297b-4c92-4d05-81c4-50f3c881816b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.011257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a1705-fccf-4a70-82bc-623a6704fbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:38.011855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a1705-fccf-4a70-82bc-623a6704fbaf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.012138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a1705-fccf-4a70-82bc-623a6704fbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.012409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a1705-fccf-4a70-82bc-623a6704fbaf None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.013070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a1705-fccf-4a70-82bc-623a6704fbaf None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:38.013372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a1705-fccf-4a70-82bc-623a6704fbaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:38.014786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a1705-fccf-4a70-82bc-623a6704fbaf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:38.022497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 252/505] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:38.023113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a1705-fccf-4a70-82bc-623a6704fbaf None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:38.058660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e297b-4c92-4d05-81c4-50f3c881816b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:38.060072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e297b-4c92-4d05-81c4-50f3c881816b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:38.062273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:38.062635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:38.125897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a1705-fccf-4a70-82bc-623a6704fbaf None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:38.154900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 254/506] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:38.155791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e297b-4c92-4d05-81c4-50f3c881816b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:38.160593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c8bce-c253-4b69-90da-4ab3fd91d532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:38.161269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c8bce-c253-4b69-90da-4ab3fd91d532 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.161616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c8bce-c253-4b69-90da-4ab3fd91d532 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.161956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c8bce-c253-4b69-90da-4ab3fd91d532 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.162780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c8bce-c253-4b69-90da-4ab3fd91d532 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:38.163086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c8bce-c253-4b69-90da-4ab3fd91d532 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:38.164492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c8bce-c253-4b69-90da-4ab3fd91d532 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:38.200222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 253/507] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 05:24:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:38.201589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cd4b5-825f-454f-ad93-b4db4e80a4bb None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.201895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cd4b5-825f-454f-ad93-b4db4e80a4bb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.202182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cd4b5-825f-454f-ad93-b4db4e80a4bb None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.203528 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 254/508] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:38.204133 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cd4b5-825f-454f-ad93-b4db4e80a4bb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:38.254706 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e297b-4c92-4d05-81c4-50f3c881816b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:38.256937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:38.288003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7710488-23df-4e4a-b667-149a8f935bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:38.288995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7710488-23df-4e4a-b667-149a8f935bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.289601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7710488-23df-4e4a-b667-149a8f935bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.290104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7710488-23df-4e4a-b667-149a8f935bb9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.297132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7710488-23df-4e4a-b667-149a8f935bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:38.297650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:38.298622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7710488-23df-4e4a-b667-149a8f935bb9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:38.305614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cd4b5-825f-454f-ad93-b4db4e80a4bb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e7710488-23df-4e4a-b667-149a8f935bb9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:38.310186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:38.311278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 255/509] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:38.312333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7710488-23df-4e4a-b667-149a8f935bb9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:38.341623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfca75c-d764-41dd-9b7e-4575ec66059d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:38.342269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfca75c-d764-41dd-9b7e-4575ec66059d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.342544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfca75c-d764-41dd-9b7e-4575ec66059d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.342817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfca75c-d764-41dd-9b7e-4575ec66059d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.352853 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfca75c-d764-41dd-9b7e-4575ec66059d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:38.354330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfca75c-d764-41dd-9b7e-4575ec66059d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fcfca75c-d764-41dd-9b7e-4575ec66059d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:38.368147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:38.369208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:38.370259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:38.370259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:38.370259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:38.370259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 255/510] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:38.370259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfca75c-d764-41dd-9b7e-4575ec66059d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:38.410912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7710488-23df-4e4a-b667-149a8f935bb9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:38.442654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99509b-3f2e-4df6-b106-a731ae340171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:38.443271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99509b-3f2e-4df6-b106-a731ae340171 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.443543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99509b-3f2e-4df6-b106-a731ae340171 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.444066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99509b-3f2e-4df6-b106-a731ae340171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.454080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99509b-3f2e-4df6-b106-a731ae340171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:38.456069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99509b-3f2e-4df6-b106-a731ae340171 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-2d99509b-3f2e-4df6-b106-a731ae340171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:38.469528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:38.471176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:38.472200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:38.472200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:38.472200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:38.472419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfca75c-d764-41dd-9b7e-4575ec66059d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:38.472504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 256/511] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:38.472504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a4fec-4c5c-4caa-961f-df86a156f8af None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.472645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a4fec-4c5c-4caa-961f-df86a156f8af None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.473045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a4fec-4c5c-4caa-961f-df86a156f8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.490625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-807a4fec-4c5c-4caa-961f-df86a156f8af None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:38.491801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-807a4fec-4c5c-4caa-961f-df86a156f8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:38.502763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:38.506397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73c6b1-bd5b-41dd-8385-3c2fce28b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:38.507001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73c6b1-bd5b-41dd-8385-3c2fce28b66a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.507279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73c6b1-bd5b-41dd-8385-3c2fce28b66a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.507552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73c6b1-bd5b-41dd-8385-3c2fce28b66a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.523378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:38.558672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 257/512] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:38.559497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a4fec-4c5c-4caa-961f-df86a156f8af None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:38.588789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73c6b1-bd5b-41dd-8385-3c2fce28b66a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:38.590307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73c6b1-bd5b-41dd-8385-3c2fce28b66a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:38.662724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a4fec-4c5c-4caa-961f-df86a156f8af None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:38.697006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 256/513] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:38.697006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73c6b1-bd5b-41dd-8385-3c2fce28b66a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:38.701604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0312c44-a554-45a0-a883-f55cfe0b6dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:38.702308 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0312c44-a554-45a0-a883-f55cfe0b6dbf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.702585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0312c44-a554-45a0-a883-f55cfe0b6dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.702848 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0312c44-a554-45a0-a883-f55cfe0b6dbf None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.703567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0312c44-a554-45a0-a883-f55cfe0b6dbf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:38.703984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0312c44-a554-45a0-a883-f55cfe0b6dbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:38.705969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0312c44-a554-45a0-a883-f55cfe0b6dbf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:38.723638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 258/514] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 05:24:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:38.724420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0312c44-a554-45a0-a883-f55cfe0b6dbf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:38.727695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:38.746377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:38.802535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73c6b1-bd5b-41dd-8385-3c2fce28b66a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:38.826095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0312c44-a554-45a0-a883-f55cfe0b6dbf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:38.838579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36046787-3fec-4076-a4aa-5c63c246e7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:38.839333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36046787-3fec-4076-a4aa-5c63c246e7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.839708 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36046787-3fec-4076-a4aa-5c63c246e7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.839980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36046787-3fec-4076-a4aa-5c63c246e7f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.841229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36046787-3fec-4076-a4aa-5c63c246e7f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:38.841536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36046787-3fec-4076-a4aa-5c63c246e7f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:38.842949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36046787-3fec-4076-a4aa-5c63c246e7f2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:38.854291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 257/515] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:38.855064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36046787-3fec-4076-a4aa-5c63c246e7f2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:38.863540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79bbf3-b76e-4d36-b717-59d5cbb60780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:38.864146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79bbf3-b76e-4d36-b717-59d5cbb60780 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.864426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79bbf3-b76e-4d36-b717-59d5cbb60780 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.864699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79bbf3-b76e-4d36-b717-59d5cbb60780 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.865340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79bbf3-b76e-4d36-b717-59d5cbb60780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:38.865641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79bbf3-b76e-4d36-b717-59d5cbb60780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:38.867032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79bbf3-b76e-4d36-b717-59d5cbb60780 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:38.879359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 259/516] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:38.880148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79bbf3-b76e-4d36-b717-59d5cbb60780 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:38.942747 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:38.955847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36046787-3fec-4076-a4aa-5c63c246e7f2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:38.963105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:38.979632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79bbf3-b76e-4d36-b717-59d5cbb60780 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:38.988944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2209f-c932-4985-9590-311a2bf53d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:38.989565 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2209f-c932-4985-9590-311a2bf53d5c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:38.989844 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2209f-c932-4985-9590-311a2bf53d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:38.990123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2209f-c932-4985-9590-311a2bf53d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:38.999996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2209f-c932-4985-9590-311a2bf53d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.002410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2209f-c932-4985-9590-311a2bf53d5c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.013090 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab9ba9-924b-4ebc-aa70-9fea5e8cfe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.013693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab9ba9-924b-4ebc-aa70-9fea5e8cfe4c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.013974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab9ba9-924b-4ebc-aa70-9fea5e8cfe4c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.014309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab9ba9-924b-4ebc-aa70-9fea5e8cfe4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-89d2209f-c932-4985-9590-311a2bf53d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:39.017255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:39.018211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:39.019070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:39.019070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:39.019070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:39.019070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 258/517] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:39.019070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2209f-c932-4985-9590-311a2bf53d5c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.083076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab9ba9-924b-4ebc-aa70-9fea5e8cfe4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.084543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab9ba9-924b-4ebc-aa70-9fea5e8cfe4c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.119171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2209f-c932-4985-9590-311a2bf53d5c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.142444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:39.152271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d985-7275-416e-9077-23e3f743c775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.152883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d985-7275-416e-9077-23e3f743c775 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.164059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d985-7275-416e-9077-23e3f743c775 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.164059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d985-7275-416e-9077-23e3f743c775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.164059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d985-7275-416e-9077-23e3f743c775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.165220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d985-7275-416e-9077-23e3f743c775 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-da36d985-7275-416e-9077-23e3f743c775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:39.179280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:39.185275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:39.186202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:39.186202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:39.186202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:39.186368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 260/518] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:39.186368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab9ba9-924b-4ebc-aa70-9fea5e8cfe4c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.186489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 259/519] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:39.186489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d985-7275-416e-9077-23e3f743c775 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.196817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:39.279084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab9ba9-924b-4ebc-aa70-9fea5e8cfe4c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.282328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d985-7275-416e-9077-23e3f743c775 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.312809 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1da2f-2a49-42fe-a1d5-0c2cb74e2475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.313432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1da2f-2a49-42fe-a1d5-0c2cb74e2475 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.313705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1da2f-2a49-42fe-a1d5-0c2cb74e2475 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.313977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1da2f-2a49-42fe-a1d5-0c2cb74e2475 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.316790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0795370-a2df-44db-ad8e-5a9858020e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.317402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0795370-a2df-44db-ad8e-5a9858020e22 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.317677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0795370-a2df-44db-ad8e-5a9858020e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.317945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0795370-a2df-44db-ad8e-5a9858020e22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.320964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1da2f-2a49-42fe-a1d5-0c2cb74e2475 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.322420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1da2f-2a49-42fe-a1d5-0c2cb74e2475 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.324766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0795370-a2df-44db-ad8e-5a9858020e22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.326177 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0795370-a2df-44db-ad8e-5a9858020e22 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4bc1da2f-2a49-42fe-a1d5-0c2cb74e2475 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:39.333530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:39.334449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 261/520] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:39.335336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1da2f-2a49-42fe-a1d5-0c2cb74e2475 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b0795370-a2df-44db-ad8e-5a9858020e22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:39.337738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:39.338584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:39.339436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:39.339436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:39.339436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:39.339436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:39.339436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:39.339436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:39.339436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:39.339436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:39.339436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 260/521] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:39.339921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0795370-a2df-44db-ad8e-5a9858020e22 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.362769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:39.406959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:39.434912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1da2f-2a49-42fe-a1d5-0c2cb74e2475 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.437192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0795370-a2df-44db-ad8e-5a9858020e22 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.468230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08423d82-2d38-4da0-be1a-b24426741fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.468850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08423d82-2d38-4da0-be1a-b24426741fad None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.469123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08423d82-2d38-4da0-be1a-b24426741fad None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.469394 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08423d82-2d38-4da0-be1a-b24426741fad None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.469950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544cde5-929a-4f3d-8ac8-ca5ad423bd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.470064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08423d82-2d38-4da0-be1a-b24426741fad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:39.470833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08423d82-2d38-4da0-be1a-b24426741fad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.471199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544cde5-929a-4f3d-8ac8-ca5ad423bd86 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.471478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544cde5-929a-4f3d-8ac8-ca5ad423bd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.471747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544cde5-929a-4f3d-8ac8-ca5ad423bd86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.472196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08423d82-2d38-4da0-be1a-b24426741fad None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.482372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544cde5-929a-4f3d-8ac8-ca5ad423bd86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.483195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 262/522] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:39.483979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08423d82-2d38-4da0-be1a-b24426741fad None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.487180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544cde5-929a-4f3d-8ac8-ca5ad423bd86 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4544cde5-929a-4f3d-8ac8-ca5ad423bd86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:39.501371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:39.502570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:39.503418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:39.503418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:39.503418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:39.507052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 261/523] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:39.507960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544cde5-929a-4f3d-8ac8-ca5ad423bd86 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.566531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:39.581638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08423d82-2d38-4da0-be1a-b24426741fad None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.607287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544cde5-929a-4f3d-8ac8-ca5ad423bd86 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.617118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b12bd-08f9-4df2-a34f-2063e11faff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.617725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b12bd-08f9-4df2-a34f-2063e11faff4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.618349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b12bd-08f9-4df2-a34f-2063e11faff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.618631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b12bd-08f9-4df2-a34f-2063e11faff4 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.619427 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b12bd-08f9-4df2-a34f-2063e11faff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:39.619731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b12bd-08f9-4df2-a34f-2063e11faff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.621091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b12bd-08f9-4df2-a34f-2063e11faff4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.629572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:39.631999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 263/524] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:39.632799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b12bd-08f9-4df2-a34f-2063e11faff4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.641481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2745f8-3889-4cb8-b670-12d94c207aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.642617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2745f8-3889-4cb8-b670-12d94c207aea None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.642907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2745f8-3889-4cb8-b670-12d94c207aea None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.643175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2745f8-3889-4cb8-b670-12d94c207aea None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.643904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2745f8-3889-4cb8-b670-12d94c207aea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:39.644204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2745f8-3889-4cb8-b670-12d94c207aea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.646073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2745f8-3889-4cb8-b670-12d94c207aea None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.665374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 262/525] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:39.666103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2745f8-3889-4cb8-b670-12d94c207aea None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.737349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b12bd-08f9-4df2-a34f-2063e11faff4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.761567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2745f8-3889-4cb8-b670-12d94c207aea None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.769460 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9ca50-6875-4f16-8b4c-e417199b8dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.770063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9ca50-6875-4f16-8b4c-e417199b8dec None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.770368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9ca50-6875-4f16-8b4c-e417199b8dec None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.770642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9ca50-6875-4f16-8b4c-e417199b8dec None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.771328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9ca50-6875-4f16-8b4c-e417199b8dec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:39.771630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9ca50-6875-4f16-8b4c-e417199b8dec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.773064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9ca50-6875-4f16-8b4c-e417199b8dec None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.790943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 264/526] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:39.791728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9ca50-6875-4f16-8b4c-e417199b8dec None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.795713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:39.800908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2335d-7a11-413f-9c63-132683fea2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.801569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2335d-7a11-413f-9c63-132683fea2af None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.801848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2335d-7a11-413f-9c63-132683fea2af None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.802119 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2335d-7a11-413f-9c63-132683fea2af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.808897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2335d-7a11-413f-9c63-132683fea2af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.810361 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2335d-7a11-413f-9c63-132683fea2af None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7ae2335d-7a11-413f-9c63-132683fea2af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:39.821580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:39.822694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:39.823530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:39.823530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:39.823530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:39.823530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:39.823530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:39.823530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:39.823530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:39.823530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:39.823530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 263/527] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:39.824348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2335d-7a11-413f-9c63-132683fea2af None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.866325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:39.895446 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9ca50-6875-4f16-8b4c-e417199b8dec None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.926780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2335d-7a11-413f-9c63-132683fea2af None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:39.928846 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fb055-6527-4b32-81fa-f83a17a66517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.929453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fb055-6527-4b32-81fa-f83a17a66517 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.929727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fb055-6527-4b32-81fa-f83a17a66517 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.929996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fb055-6527-4b32-81fa-f83a17a66517 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.930686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fb055-6527-4b32-81fa-f83a17a66517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:39.930986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fb055-6527-4b32-81fa-f83a17a66517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.932355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fb055-6527-4b32-81fa-f83a17a66517 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:39.943524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 265/528] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:39.944555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fb055-6527-4b32-81fa-f83a17a66517 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:39.961356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3e6aa-966e-4ba3-b459-800bd97d7b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:39.962158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3e6aa-966e-4ba3-b459-800bd97d7b65 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:39.962609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3e6aa-966e-4ba3-b459-800bd97d7b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:39.962885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3e6aa-966e-4ba3-b459-800bd97d7b65 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:39.963524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3e6aa-966e-4ba3-b459-800bd97d7b65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:39.964947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3e6aa-966e-4ba3-b459-800bd97d7b65 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:40.003540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:40.038858 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fb055-6527-4b32-81fa-f83a17a66517 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:40.059371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 264/529] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:24:39 2022] POST /identity/v3/users => generated 247 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:24:40.060183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3e6aa-966e-4ba3-b459-800bd97d7b65 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:40.073523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e7a27-07d1-4874-99fc-217a1303ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:40.074520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e7a27-07d1-4874-99fc-217a1303ca52 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:40.075094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e7a27-07d1-4874-99fc-217a1303ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:40.075371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e7a27-07d1-4874-99fc-217a1303ca52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:40.154899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e7a27-07d1-4874-99fc-217a1303ca52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:40.156406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e7a27-07d1-4874-99fc-217a1303ca52 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:40.170070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:40.177289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3e6aa-966e-4ba3-b459-800bd97d7b65 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:40.218860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa6596-945a-40ca-92ae-e8aa9d466596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:40.219481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa6596-945a-40ca-92ae-e8aa9d466596 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:40.219760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa6596-945a-40ca-92ae-e8aa9d466596 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:40.220046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa6596-945a-40ca-92ae-e8aa9d466596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:40.220799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa6596-945a-40ca-92ae-e8aa9d466596 None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=c34eb8fcef0543c282b3887dfeb74c15) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:40.221106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa6596-945a-40ca-92ae-e8aa9d466596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:40.222582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa6596-945a-40ca-92ae-e8aa9d466596 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:40.223074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:40.238168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 265/530] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:40 2022] GET /identity/v3/role_assignments?user.id=c34eb8fcef0543c282b3887dfeb74c15&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:40.238168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa6596-945a-40ca-92ae-e8aa9d466596 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:40.284216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 266/531] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:40.285366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e7a27-07d1-4874-99fc-217a1303ca52 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:40.373942 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa6596-945a-40ca-92ae-e8aa9d466596 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:40.416399 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccbcbe-9352-4d28-b129-3bdd6f5cc49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:40.417022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccbcbe-9352-4d28-b129-3bdd6f5cc49b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:40.417303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccbcbe-9352-4d28-b129-3bdd6f5cc49b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:40.417578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccbcbe-9352-4d28-b129-3bdd6f5cc49b None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:40.418309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccbcbe-9352-4d28-b129-3bdd6f5cc49b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:40.418792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccbcbe-9352-4d28-b129-3bdd6f5cc49b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:40.421464 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccbcbe-9352-4d28-b129-3bdd6f5cc49b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:40.427977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e7a27-07d1-4874-99fc-217a1303ca52 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:40.449541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 266/532] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:40.449541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccbcbe-9352-4d28-b129-3bdd6f5cc49b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:40.469852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0c1d6-571d-4946-a394-4bf66e2e7388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:40.470701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0c1d6-571d-4946-a394-4bf66e2e7388 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:40.471158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0c1d6-571d-4946-a394-4bf66e2e7388 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:40.471512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0c1d6-571d-4946-a394-4bf66e2e7388 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:40.475639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:40.480044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0c1d6-571d-4946-a394-4bf66e2e7388 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:40.481799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0c1d6-571d-4946-a394-4bf66e2e7388 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b1d0c1d6-571d-4946-a394-4bf66e2e7388 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:40.496355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:40.503555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:40.504417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:40.504417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:40.504417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:40.504417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:40.504417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:40.504417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:40.504417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:40.504417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:40.504417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 267/533] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:40.504417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0c1d6-571d-4946-a394-4bf66e2e7388 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:40.534074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:40.588110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccbcbe-9352-4d28-b129-3bdd6f5cc49b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:40.624450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968e5e8-f1f1-4aa4-abc9-287357c5125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:40.625071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968e5e8-f1f1-4aa4-abc9-287357c5125d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:40.625353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968e5e8-f1f1-4aa4-abc9-287357c5125d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:40.625624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968e5e8-f1f1-4aa4-abc9-287357c5125d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:40.634871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0c1d6-571d-4946-a394-4bf66e2e7388 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:40.637100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968e5e8-f1f1-4aa4-abc9-287357c5125d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:40.639141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968e5e8-f1f1-4aa4-abc9-287357c5125d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1968e5e8-f1f1-4aa4-abc9-287357c5125d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:40.653747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:40.654729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:40.655579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:40.655579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:40.655579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:40.659512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 267/534] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:40.660366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968e5e8-f1f1-4aa4-abc9-287357c5125d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:40.681502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a088ef-3567-42c8-bc00-1b4092711452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:40.682489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a088ef-3567-42c8-bc00-1b4092711452 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:40.682976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a088ef-3567-42c8-bc00-1b4092711452 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:40.683451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a088ef-3567-42c8-bc00-1b4092711452 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:40.684975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a088ef-3567-42c8-bc00-1b4092711452 None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=fba4f0473f4c4d3bb2d50b09f3de9837) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:40.685780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a088ef-3567-42c8-bc00-1b4092711452 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:40.688162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a088ef-3567-42c8-bc00-1b4092711452 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:40.697649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 268/535] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:40 2022] GET /identity/v3/role_assignments?user.id=fba4f0473f4c4d3bb2d50b09f3de9837&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:40.698770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a088ef-3567-42c8-bc00-1b4092711452 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:40.774523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:40.843152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968e5e8-f1f1-4aa4-abc9-287357c5125d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:40.866887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:40.885546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74997513-08fb-4476-a67e-53bf2a427690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:40.886199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74997513-08fb-4476-a67e-53bf2a427690 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:40.886486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74997513-08fb-4476-a67e-53bf2a427690 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:40.886762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74997513-08fb-4476-a67e-53bf2a427690 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:40.887471 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74997513-08fb-4476-a67e-53bf2a427690 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:40.887780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74997513-08fb-4476-a67e-53bf2a427690 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:40.889226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74997513-08fb-4476-a67e-53bf2a427690 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:40.892242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a088ef-3567-42c8-bc00-1b4092711452 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:40.904409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 268/536] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:40.913953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74997513-08fb-4476-a67e-53bf2a427690 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:40.971984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc6604-84e8-40d1-86ef-ae5ec8a25771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:40.972524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc6604-84e8-40d1-86ef-ae5ec8a25771 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:40.972856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc6604-84e8-40d1-86ef-ae5ec8a25771 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:40.973128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc6604-84e8-40d1-86ef-ae5ec8a25771 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:40.973824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc6604-84e8-40d1-86ef-ae5ec8a25771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:40.974132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc6604-84e8-40d1-86ef-ae5ec8a25771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:40.975531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc6604-84e8-40d1-86ef-ae5ec8a25771 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:40.992992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 269/537] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:40.994272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc6604-84e8-40d1-86ef-ae5ec8a25771 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:41.018344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74997513-08fb-4476-a67e-53bf2a427690 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:41.051634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:41.060932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fa972-e296-4a75-b736-cb58ff81942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:41.061580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fa972-e296-4a75-b736-cb58ff81942c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:41.063115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fa972-e296-4a75-b736-cb58ff81942c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:41.063115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fa972-e296-4a75-b736-cb58ff81942c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:41.063115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fa972-e296-4a75-b736-cb58ff81942c None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=131afaf047d34e88842fffd0d0b76242) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:41.063336 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fa972-e296-4a75-b736-cb58ff81942c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:41.064831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fa972-e296-4a75-b736-cb58ff81942c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:41.075753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 269/538] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:40 2022] GET /identity/v3/role_assignments?user.id=131afaf047d34e88842fffd0d0b76242&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:41.172405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc6604-84e8-40d1-86ef-ae5ec8a25771 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:41.218122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd56bfb-bf2a-4fb1-a389-fb5dbabda756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:41.218807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd56bfb-bf2a-4fb1-a389-fb5dbabda756 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:41.219089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd56bfb-bf2a-4fb1-a389-fb5dbabda756 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:41.219366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd56bfb-bf2a-4fb1-a389-fb5dbabda756 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:41.229900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd56bfb-bf2a-4fb1-a389-fb5dbabda756 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:41.231329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd56bfb-bf2a-4fb1-a389-fb5dbabda756 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8bd56bfb-bf2a-4fb1-a389-fb5dbabda756 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:41.244341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:41.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:41.246102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:41.246102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:41.246102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:41.246102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:41.246102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:41.246102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:41.246102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:41.246102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:41.246102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 270/539] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:41.253908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fa972-e296-4a75-b736-cb58ff81942c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:41.365674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fa972-e296-4a75-b736-cb58ff81942c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:41.400924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e172d4d-934f-459e-a3d0-9dd2da194f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:41.401535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172d4d-934f-459e-a3d0-9dd2da194f82 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:41.401812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172d4d-934f-459e-a3d0-9dd2da194f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:41.402084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172d4d-934f-459e-a3d0-9dd2da194f82 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:41.402867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e172d4d-934f-459e-a3d0-9dd2da194f82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:41.403176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e172d4d-934f-459e-a3d0-9dd2da194f82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:41.404731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e172d4d-934f-459e-a3d0-9dd2da194f82 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:41.416813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 270/540] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:41.422530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd56bfb-bf2a-4fb1-a389-fb5dbabda756 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:41.434509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:41.560956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd56bfb-bf2a-4fb1-a389-fb5dbabda756 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:41.612327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24f60a-750e-4765-b0a9-eda29c4832ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:41.612954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24f60a-750e-4765-b0a9-eda29c4832ca None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:41.613232 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24f60a-750e-4765-b0a9-eda29c4832ca None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:41.613499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24f60a-750e-4765-b0a9-eda29c4832ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:41.614259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24f60a-750e-4765-b0a9-eda29c4832ca None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=3a0feaf83ee14071806f36d01c36b635) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:41.614566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24f60a-750e-4765-b0a9-eda29c4832ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:41.616014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24f60a-750e-4765-b0a9-eda29c4832ca None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:41.622156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 271/541] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:41 2022] GET /identity/v3/role_assignments?user.id=3a0feaf83ee14071806f36d01c36b635&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:42.678204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10220-d87c-444c-9f5d-7828904cb59e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:42.678718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10220-d87c-444c-9f5d-7828904cb59e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:42.678794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de10220-d87c-444c-9f5d-7828904cb59e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:42.680333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 271/542] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:42.685868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af976464-639d-4907-9a50-40e91f929770 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:42.686279 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af976464-639d-4907-9a50-40e91f929770 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:42.686442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af976464-639d-4907-9a50-40e91f929770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:42.715083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-af976464-639d-4907-9a50-40e91f929770 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:42.716304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-af976464-639d-4907-9a50-40e91f929770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:42.794672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 272/543] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:42.952744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bca7b-15f0-4d90-b1e8-3b913109306f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:42.953329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bca7b-15f0-4d90-b1e8-3b913109306f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:42.953799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bca7b-15f0-4d90-b1e8-3b913109306f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:42.955722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 272/544] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:42.960631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe101df-a39f-4c40-b661-868fccfd486c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:42.960925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe101df-a39f-4c40-b661-868fccfd486c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:42.961203 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe101df-a39f-4c40-b661-868fccfd486c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:42.971318 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:42.987138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9fe101df-a39f-4c40-b661-868fccfd486c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:42.989539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9fe101df-a39f-4c40-b661-868fccfd486c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:43.071639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 273/545] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:43.175172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bca7b-15f0-4d90-b1e8-3b913109306f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:43.235608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:43.281748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bca7b-15f0-4d90-b1e8-3b913109306f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:43.329587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63079d-9461-4235-8860-6a089fb37951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:43.330270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63079d-9461-4235-8860-6a089fb37951 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:43.330589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63079d-9461-4235-8860-6a089fb37951 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:43.330831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63079d-9461-4235-8860-6a089fb37951 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:43.347659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63079d-9461-4235-8860-6a089fb37951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1b63079d-9461-4235-8860-6a089fb37951 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:43.352994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:43.353858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:43.355092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 273/546] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:43.360221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe101df-a39f-4c40-b661-868fccfd486c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:43.421266 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00547da-2b99-419c-a6ba-e6f07ae48519 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:43.421606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00547da-2b99-419c-a6ba-e6f07ae48519 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:43.421816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00547da-2b99-419c-a6ba-e6f07ae48519 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:43.423467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 274/547] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:24:43.431891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec86f23-6470-4406-a42e-9fc5161d3e7d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:43.432178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec86f23-6470-4406-a42e-9fc5161d3e7d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:43.432448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec86f23-6470-4406-a42e-9fc5161d3e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:43.453399 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7ec86f23-6470-4406-a42e-9fc5161d3e7d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:43.455157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7ec86f23-6470-4406-a42e-9fc5161d3e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:43.507425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe101df-a39f-4c40-b661-868fccfd486c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:43.546219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 275/548] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:43.547432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec86f23-6470-4406-a42e-9fc5161d3e7d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:43.558510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:43.565732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57edecd-01f0-46bb-aa59-588994ba052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:43.566685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57edecd-01f0-46bb-aa59-588994ba052d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:43.567129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57edecd-01f0-46bb-aa59-588994ba052d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:43.572385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57edecd-01f0-46bb-aa59-588994ba052d None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:43.572385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57edecd-01f0-46bb-aa59-588994ba052d None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:43.572385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57edecd-01f0-46bb-aa59-588994ba052d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:43.572385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57edecd-01f0-46bb-aa59-588994ba052d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:43.590028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 274/549] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:43.596575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57edecd-01f0-46bb-aa59-588994ba052d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:43.627587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:43.687870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec86f23-6470-4406-a42e-9fc5161d3e7d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:43.727615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7840250-4574-47d5-9fb7-83734688b066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:43.728368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7840250-4574-47d5-9fb7-83734688b066 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:43.728647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7840250-4574-47d5-9fb7-83734688b066 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:43.728918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7840250-4574-47d5-9fb7-83734688b066 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:43.729967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57edecd-01f0-46bb-aa59-588994ba052d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:43.746875 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7840250-4574-47d5-9fb7-83734688b066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c7840250-4574-47d5-9fb7-83734688b066 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:43.753512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:43.754424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:43.755794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 276/550] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:43.757284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccc4a8-6f35-4ad8-b68f-8d6b27ea2dcf None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:43.757382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccc4a8-6f35-4ad8-b68f-8d6b27ea2dcf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:43.759474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccc4a8-6f35-4ad8-b68f-8d6b27ea2dcf None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:43.759474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 277/551] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:43.760219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccc4a8-6f35-4ad8-b68f-8d6b27ea2dcf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:43.779854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c0f77-0b4e-4513-bfdb-a571825d24a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:43.780484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c0f77-0b4e-4513-bfdb-a571825d24a4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:43.780809 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c0f77-0b4e-4513-bfdb-a571825d24a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:43.781087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c0f77-0b4e-4513-bfdb-a571825d24a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:43.792048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c0f77-0b4e-4513-bfdb-a571825d24a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:43.793556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c0f77-0b4e-4513-bfdb-a571825d24a4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5f1c0f77-0b4e-4513-bfdb-a571825d24a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:43.808078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:43.810196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:43.812269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:43.812269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:43.812269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:43.812269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 275/552] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:43.812269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bcd6f-193a-40fc-9288-f4c09ad2a86f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:43.812269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bcd6f-193a-40fc-9288-f4c09ad2a86f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:43.812269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bcd6f-193a-40fc-9288-f4c09ad2a86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:43.832538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ad4bcd6f-193a-40fc-9288-f4c09ad2a86f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:43.833735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ad4bcd6f-193a-40fc-9288-f4c09ad2a86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:43.877811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:43.884847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccc4a8-6f35-4ad8-b68f-8d6b27ea2dcf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:43.898716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:43.918166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 276/553] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:43.918166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bcd6f-193a-40fc-9288-f4c09ad2a86f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:43.932631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102550b-e1a3-4659-aad5-737778af14fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:43.933404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102550b-e1a3-4659-aad5-737778af14fe None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:43.933937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102550b-e1a3-4659-aad5-737778af14fe None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:43.934277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102550b-e1a3-4659-aad5-737778af14fe None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:43.935079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102550b-e1a3-4659-aad5-737778af14fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:43.935682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102550b-e1a3-4659-aad5-737778af14fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:43.937339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102550b-e1a3-4659-aad5-737778af14fe None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:43.947317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 278/554] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:43.948328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102550b-e1a3-4659-aad5-737778af14fe None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.051091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bcd6f-193a-40fc-9288-f4c09ad2a86f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.060454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102550b-e1a3-4659-aad5-737778af14fe None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.096601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a47ef-2175-40f4-9741-3cd93e65b043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.097241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a47ef-2175-40f4-9741-3cd93e65b043 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.097611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a47ef-2175-40f4-9741-3cd93e65b043 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.097956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a47ef-2175-40f4-9741-3cd93e65b043 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.098737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a47ef-2175-40f4-9741-3cd93e65b043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:44.099042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a47ef-2175-40f4-9741-3cd93e65b043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:44.100499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12788d6-8879-45d3-845d-6a797ca6bc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.100639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a47ef-2175-40f4-9741-3cd93e65b043 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:44.101196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12788d6-8879-45d3-845d-6a797ca6bc8f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.101478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12788d6-8879-45d3-845d-6a797ca6bc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.101753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12788d6-8879-45d3-845d-6a797ca6bc8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.114896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12788d6-8879-45d3-845d-6a797ca6bc8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:44.119219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 277/555] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:44.120076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a47ef-2175-40f4-9741-3cd93e65b043 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c12788d6-8879-45d3-845d-6a797ca6bc8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:44.120434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:44.121388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:44.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:44.126978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 279/556] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:44.127706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12788d6-8879-45d3-845d-6a797ca6bc8f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.160641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:44.161569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:44.240190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a47ef-2175-40f4-9741-3cd93e65b043 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.255070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12788d6-8879-45d3-845d-6a797ca6bc8f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.276958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8b550-b234-44ee-aa9b-aa8152d3f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.277677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b550-b234-44ee-aa9b-aa8152d3f9cb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.278026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b550-b234-44ee-aa9b-aa8152d3f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.278398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8b550-b234-44ee-aa9b-aa8152d3f9cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.288582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8b550-b234-44ee-aa9b-aa8152d3f9cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:44.290067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8b550-b234-44ee-aa9b-aa8152d3f9cb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:44.291253 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17338a5d-21aa-424f-bcb8-c859f6f84b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.291887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17338a5d-21aa-424f-bcb8-c859f6f84b8d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.292171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17338a5d-21aa-424f-bcb8-c859f6f84b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.292447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17338a5d-21aa-424f-bcb8-c859f6f84b8d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.300179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17338a5d-21aa-424f-bcb8-c859f6f84b8d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:44.301675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17338a5d-21aa-424f-bcb8-c859f6f84b8d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-fff8b550-b234-44ee-aa9b-aa8152d3f9cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:44.305917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:44.306926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:44.307799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:44.307799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:44.307799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:44.309518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 278/557] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:44.310628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8b550-b234-44ee-aa9b-aa8152d3f9cb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-17338a5d-21aa-424f-bcb8-c859f6f84b8d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:44.313735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:44.315842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 280/558] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:44.317012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17338a5d-21aa-424f-bcb8-c859f6f84b8d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.413244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:44.418006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:44.426911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8b550-b234-44ee-aa9b-aa8152d3f9cb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.431247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17338a5d-21aa-424f-bcb8-c859f6f84b8d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.460947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dd50e-7d37-44c0-8ee5-66e603db2cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.461556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dd50e-7d37-44c0-8ee5-66e603db2cfa None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.461837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dd50e-7d37-44c0-8ee5-66e603db2cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.462111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dd50e-7d37-44c0-8ee5-66e603db2cfa None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.462805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dd50e-7d37-44c0-8ee5-66e603db2cfa None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:44.463108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dd50e-7d37-44c0-8ee5-66e603db2cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:44.464496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dd50e-7d37-44c0-8ee5-66e603db2cfa None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:44.465370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0713ee-8e37-4a52-89b7-6dd83b52e6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.466110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0713ee-8e37-4a52-89b7-6dd83b52e6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.466422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0713ee-8e37-4a52-89b7-6dd83b52e6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.466702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0713ee-8e37-4a52-89b7-6dd83b52e6f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.472552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 279/559] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:44.473331 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dd50e-7d37-44c0-8ee5-66e603db2cfa None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.478317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0713ee-8e37-4a52-89b7-6dd83b52e6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-da0713ee-8e37-4a52-89b7-6dd83b52e6f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:44.483178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:44.484290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 281/560] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:44.486539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0713ee-8e37-4a52-89b7-6dd83b52e6f2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.572420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dd50e-7d37-44c0-8ee5-66e603db2cfa None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.582898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0713ee-8e37-4a52-89b7-6dd83b52e6f2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.604932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb068a98-d838-4268-bfc8-fd952203d53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.605533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb068a98-d838-4268-bfc8-fd952203d53e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.605811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb068a98-d838-4268-bfc8-fd952203d53e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.606083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb068a98-d838-4268-bfc8-fd952203d53e None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.606741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb068a98-d838-4268-bfc8-fd952203d53e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:44.607046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb068a98-d838-4268-bfc8-fd952203d53e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:44.608402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb068a98-d838-4268-bfc8-fd952203d53e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:44.617160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f55576-ca25-40fe-b51e-215c678d36c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.617753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f55576-ca25-40fe-b51e-215c678d36c7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.618026 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f55576-ca25-40fe-b51e-215c678d36c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.618326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f55576-ca25-40fe-b51e-215c678d36c7 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.619043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f55576-ca25-40fe-b51e-215c678d36c7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:44.619343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f55576-ca25-40fe-b51e-215c678d36c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:44.620122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 280/561] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:44.620738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f55576-ca25-40fe-b51e-215c678d36c7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:44.621358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1551f24-17e0-4d15-9568-62acc35eae3b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.621641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1551f24-17e0-4d15-9568-62acc35eae3b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.621913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1551f24-17e0-4d15-9568-62acc35eae3b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.623131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 281/562] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:44.623814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1551f24-17e0-4d15-9568-62acc35eae3b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.635655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:44.641233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 282/563] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:44.642087 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f55576-ca25-40fe-b51e-215c678d36c7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.651719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:44.724006 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1551f24-17e0-4d15-9568-62acc35eae3b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.743013 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f55576-ca25-40fe-b51e-215c678d36c7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.757075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31140ad1-aac6-4506-b7e7-328876569800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.757709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31140ad1-aac6-4506-b7e7-328876569800 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.757988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31140ad1-aac6-4506-b7e7-328876569800 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.758291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31140ad1-aac6-4506-b7e7-328876569800 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.768588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31140ad1-aac6-4506-b7e7-328876569800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:44.770046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31140ad1-aac6-4506-b7e7-328876569800 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:44.776153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007699a3-287f-4bdb-8911-798f145fc911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.776749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007699a3-287f-4bdb-8911-798f145fc911 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.777025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007699a3-287f-4bdb-8911-798f145fc911 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.777300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007699a3-287f-4bdb-8911-798f145fc911 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.777945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007699a3-287f-4bdb-8911-798f145fc911 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:44.778275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007699a3-287f-4bdb-8911-798f145fc911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:44.779616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007699a3-287f-4bdb-8911-798f145fc911 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-31140ad1-aac6-4506-b7e7-328876569800 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:44.785078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:44.786007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:44.787552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:44.787552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:44.787552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:44.787552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 282/564] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:44.787552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31140ad1-aac6-4506-b7e7-328876569800 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.787843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 283/565] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:44.788609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6ee3f-7279-4757-ac25-e48ebccc7dbe None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.788885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6ee3f-7279-4757-ac25-e48ebccc7dbe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.789152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6ee3f-7279-4757-ac25-e48ebccc7dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.806600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f2c6ee3f-7279-4757-ac25-e48ebccc7dbe None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:44.807767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f2c6ee3f-7279-4757-ac25-e48ebccc7dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:44.851165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:44.868713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:44.873959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 284/566] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:44.874785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6ee3f-7279-4757-ac25-e48ebccc7dbe None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:44.894382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31140ad1-aac6-4506-b7e7-328876569800 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:44.930278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23569584-d029-4d1a-9746-ed6627d7328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:44.930912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23569584-d029-4d1a-9746-ed6627d7328d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:44.931188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23569584-d029-4d1a-9746-ed6627d7328d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:44.931457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23569584-d029-4d1a-9746-ed6627d7328d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:44.939262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23569584-d029-4d1a-9746-ed6627d7328d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:44.940705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23569584-d029-4d1a-9746-ed6627d7328d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-23569584-d029-4d1a-9746-ed6627d7328d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:44.951896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:44.952833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 283/567] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:44.953750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23569584-d029-4d1a-9746-ed6627d7328d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.004390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6ee3f-7279-4757-ac25-e48ebccc7dbe None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.044513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117891e-bd90-4c2f-91c5-e60773d72a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.045116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117891e-bd90-4c2f-91c5-e60773d72a3f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.045501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117891e-bd90-4c2f-91c5-e60773d72a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.045771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117891e-bd90-4c2f-91c5-e60773d72a3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.054574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23569584-d029-4d1a-9746-ed6627d7328d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.080431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:45.092793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed79f6-fadb-4e18-93b7-66c1d9f435d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.093391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed79f6-fadb-4e18-93b7-66c1d9f435d5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.093668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed79f6-fadb-4e18-93b7-66c1d9f435d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.094173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed79f6-fadb-4e18-93b7-66c1d9f435d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.094602 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed79f6-fadb-4e18-93b7-66c1d9f435d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:45.094901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed79f6-fadb-4e18-93b7-66c1d9f435d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.096817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed79f6-fadb-4e18-93b7-66c1d9f435d5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.108267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 284/568] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:45.108999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed79f6-fadb-4e18-93b7-66c1d9f435d5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.120523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:45.124977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117891e-bd90-4c2f-91c5-e60773d72a3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.126668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117891e-bd90-4c2f-91c5-e60773d72a3f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.207449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed79f6-fadb-4e18-93b7-66c1d9f435d5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.224674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 285/569] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:45.225513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117891e-bd90-4c2f-91c5-e60773d72a3f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.243011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2cc28-e181-4dbe-ba61-7f5ab6336d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.243638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2cc28-e181-4dbe-ba61-7f5ab6336d8f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.243912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2cc28-e181-4dbe-ba61-7f5ab6336d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.244181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2cc28-e181-4dbe-ba61-7f5ab6336d8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.255018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2cc28-e181-4dbe-ba61-7f5ab6336d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.256439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2cc28-e181-4dbe-ba61-7f5ab6336d8f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-09f2cc28-e181-4dbe-ba61-7f5ab6336d8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:45.271219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:45.272247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:45.273130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:45.273130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:45.273130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:45.273130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 285/570] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:45.273130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2cc28-e181-4dbe-ba61-7f5ab6336d8f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.308417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:45.319277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:45.322712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117891e-bd90-4c2f-91c5-e60773d72a3f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.355414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439759f-d790-4d40-97d0-d089413e4fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.356009 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439759f-d790-4d40-97d0-d089413e4fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.356280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439759f-d790-4d40-97d0-d089413e4fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.356543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439759f-d790-4d40-97d0-d089413e4fa5 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.357251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439759f-d790-4d40-97d0-d089413e4fa5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:45.357550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439759f-d790-4d40-97d0-d089413e4fa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.359035 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439759f-d790-4d40-97d0-d089413e4fa5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.373108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2cc28-e181-4dbe-ba61-7f5ab6336d8f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.377007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 286/571] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:45.377796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439759f-d790-4d40-97d0-d089413e4fa5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.408615 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d69d39-d05d-4ba0-8b81-e66bf6c31972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.409252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d69d39-d05d-4ba0-8b81-e66bf6c31972 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.409554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d69d39-d05d-4ba0-8b81-e66bf6c31972 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.409823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d69d39-d05d-4ba0-8b81-e66bf6c31972 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.416904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d69d39-d05d-4ba0-8b81-e66bf6c31972 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.418342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d69d39-d05d-4ba0-8b81-e66bf6c31972 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-61d69d39-d05d-4ba0-8b81-e66bf6c31972 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:45.429445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:45.430526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 286/572] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:45.431816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d69d39-d05d-4ba0-8b81-e66bf6c31972 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.477883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439759f-d790-4d40-97d0-d089413e4fa5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.511605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078221c-2b99-41e5-a638-9b900d90d4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.512222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078221c-2b99-41e5-a638-9b900d90d4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.512520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078221c-2b99-41e5-a638-9b900d90d4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.512785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078221c-2b99-41e5-a638-9b900d90d4e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.520675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:45.523319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078221c-2b99-41e5-a638-9b900d90d4e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:45.527158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d69d39-d05d-4ba0-8b81-e66bf6c31972 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0078221c-2b99-41e5-a638-9b900d90d4e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:45.532859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:45.533940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 287/573] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:45.534854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078221c-2b99-41e5-a638-9b900d90d4e3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.548587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:45.561083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e541b35-3e40-412b-b27f-9da8c9d7825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.561714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e541b35-3e40-412b-b27f-9da8c9d7825d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.561992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e541b35-3e40-412b-b27f-9da8c9d7825d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.562298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e541b35-3e40-412b-b27f-9da8c9d7825d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.572711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e541b35-3e40-412b-b27f-9da8c9d7825d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.574165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e541b35-3e40-412b-b27f-9da8c9d7825d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-1e541b35-3e40-412b-b27f-9da8c9d7825d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:45.589401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:45.596102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:45.597122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:45.597122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:45.597122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:45.597122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 287/574] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:45.597122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e541b35-3e40-412b-b27f-9da8c9d7825d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.631867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078221c-2b99-41e5-a638-9b900d90d4e3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.664175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb22c3-babd-42fc-a001-8d39d0330280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.664775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb22c3-babd-42fc-a001-8d39d0330280 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.665063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb22c3-babd-42fc-a001-8d39d0330280 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.665339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb22c3-babd-42fc-a001-8d39d0330280 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.665977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb22c3-babd-42fc-a001-8d39d0330280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:45.666322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb22c3-babd-42fc-a001-8d39d0330280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.667685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb22c3-babd-42fc-a001-8d39d0330280 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.678991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 288/575] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:45.679775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb22c3-babd-42fc-a001-8d39d0330280 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.691535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e541b35-3e40-412b-b27f-9da8c9d7825d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.724075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7438260-77e2-47ff-95cb-53d3d2f9d0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.724701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7438260-77e2-47ff-95cb-53d3d2f9d0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.724978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7438260-77e2-47ff-95cb-53d3d2f9d0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.725251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7438260-77e2-47ff-95cb-53d3d2f9d0c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.735420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7438260-77e2-47ff-95cb-53d3d2f9d0c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.737378 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7438260-77e2-47ff-95cb-53d3d2f9d0c5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.743422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e7438260-77e2-47ff-95cb-53d3d2f9d0c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:45.754283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:45.760193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:45.761287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:45.761287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:45.761287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:45.761502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:45.761854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 288/576] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:45.761854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7438260-77e2-47ff-95cb-53d3d2f9d0c5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.778773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb22c3-babd-42fc-a001-8d39d0330280 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.811689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b61e7f-68a8-4796-ba68-967d73e738e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.812287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b61e7f-68a8-4796-ba68-967d73e738e8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.812567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b61e7f-68a8-4796-ba68-967d73e738e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.812841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b61e7f-68a8-4796-ba68-967d73e738e8 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.813560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b61e7f-68a8-4796-ba68-967d73e738e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:45.814506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b61e7f-68a8-4796-ba68-967d73e738e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.815913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b61e7f-68a8-4796-ba68-967d73e738e8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.832929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 289/577] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:45.833709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b61e7f-68a8-4796-ba68-967d73e738e8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.859614 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7438260-77e2-47ff-95cb-53d3d2f9d0c5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.894128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ccc16-7f46-4ca2-b9aa-985950d8a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.894761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ccc16-7f46-4ca2-b9aa-985950d8a64e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.895039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ccc16-7f46-4ca2-b9aa-985950d8a64e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.895312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ccc16-7f46-4ca2-b9aa-985950d8a64e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.895975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ccc16-7f46-4ca2-b9aa-985950d8a64e None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:45.896278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ccc16-7f46-4ca2-b9aa-985950d8a64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.897655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ccc16-7f46-4ca2-b9aa-985950d8a64e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.905724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 289/578] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:45.906608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ccc16-7f46-4ca2-b9aa-985950d8a64e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.931094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b61e7f-68a8-4796-ba68-967d73e738e8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:45.970989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebceb8-d05f-49cb-b33a-c7da534b185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:45.971585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebceb8-d05f-49cb-b33a-c7da534b185f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:45.971862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebceb8-d05f-49cb-b33a-c7da534b185f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:45.972138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebceb8-d05f-49cb-b33a-c7da534b185f None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:45.972767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebceb8-d05f-49cb-b33a-c7da534b185f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:45.973066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebceb8-d05f-49cb-b33a-c7da534b185f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:45.974453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebceb8-d05f-49cb-b33a-c7da534b185f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:45.975822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:45.985489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 290/579] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:45.986771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebceb8-d05f-49cb-b33a-c7da534b185f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:45.994476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:46.035056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ccc16-7f46-4ca2-b9aa-985950d8a64e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.067995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2a642-83c2-4469-b5a9-c619488333cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.068624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2a642-83c2-4469-b5a9-c619488333cf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.069150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2a642-83c2-4469-b5a9-c619488333cf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.069422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2a642-83c2-4469-b5a9-c619488333cf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.076721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2a642-83c2-4469-b5a9-c619488333cf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.078219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2a642-83c2-4469-b5a9-c619488333cf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.083861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebceb8-d05f-49cb-b33a-c7da534b185f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0fb2a642-83c2-4469-b5a9-c619488333cf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:46.089024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:46.090104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:46.091391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:46.091391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:46.091391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:46.091391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:46.091391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:46.091391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:46.091391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:46.091391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:46.091391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 05:24:46.091391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:46.094276 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 290/580] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:46.095764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2a642-83c2-4469-b5a9-c619488333cf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.117633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae2e9-ad37-482e-89c2-66f0aac673f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.118259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae2e9-ad37-482e-89c2-66f0aac673f5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.118536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae2e9-ad37-482e-89c2-66f0aac673f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.118804 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae2e9-ad37-482e-89c2-66f0aac673f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.119434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ae2e9-ad37-482e-89c2-66f0aac673f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:46.119731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ae2e9-ad37-482e-89c2-66f0aac673f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.121078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ae2e9-ad37-482e-89c2-66f0aac673f5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.133144 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 291/581] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:46.134127 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae2e9-ad37-482e-89c2-66f0aac673f5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.186121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:46.195207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2a642-83c2-4469-b5a9-c619488333cf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.214527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:46.227692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccb61a-ae19-4625-b72e-792f1c67dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.228317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccb61a-ae19-4625-b72e-792f1c67dbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.228595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccb61a-ae19-4625-b72e-792f1c67dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.228866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccb61a-ae19-4625-b72e-792f1c67dbf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.229830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae2e9-ad37-482e-89c2-66f0aac673f5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.239218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccb61a-ae19-4625-b72e-792f1c67dbf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.240663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccb61a-ae19-4625-b72e-792f1c67dbf3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f8ccb61a-ae19-4625-b72e-792f1c67dbf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:46.255170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:46.256149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:46.257012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:46.257012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:46.257012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:46.257012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 291/582] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:46.257012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccb61a-ae19-4625-b72e-792f1c67dbf3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.266176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63169174-041c-415e-a75f-613fba2b7852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.266836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63169174-041c-415e-a75f-613fba2b7852 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.267629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63169174-041c-415e-a75f-613fba2b7852 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.267901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63169174-041c-415e-a75f-613fba2b7852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.278171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63169174-041c-415e-a75f-613fba2b7852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.279583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63169174-041c-415e-a75f-613fba2b7852 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-63169174-041c-415e-a75f-613fba2b7852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:46.293438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:46.294594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:46.295656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:46.295656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:46.295656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:46.295656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 292/583] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:46.295656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63169174-041c-415e-a75f-613fba2b7852 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.356953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccb61a-ae19-4625-b72e-792f1c67dbf3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.389971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d0aa2-e240-4d2a-8a73-7d798e91c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.390619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d0aa2-e240-4d2a-8a73-7d798e91c075 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.390893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d0aa2-e240-4d2a-8a73-7d798e91c075 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.391162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d0aa2-e240-4d2a-8a73-7d798e91c075 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.393081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63169174-041c-415e-a75f-613fba2b7852 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.397565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d0aa2-e240-4d2a-8a73-7d798e91c075 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.399017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d0aa2-e240-4d2a-8a73-7d798e91c075 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.400208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-320d0aa2-e240-4d2a-8a73-7d798e91c075 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:46.410075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:46.416509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:46.417606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:46.417606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:46.417606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:46.417606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:46.417606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:46.417606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:46.417606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:46.417606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:46.417606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 292/584] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:46.417606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d0aa2-e240-4d2a-8a73-7d798e91c075 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.426618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bc630-459a-4523-8108-ac4702fcb412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.427344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bc630-459a-4523-8108-ac4702fcb412 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.427498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bc630-459a-4523-8108-ac4702fcb412 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.427768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bc630-459a-4523-8108-ac4702fcb412 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.428481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bc630-459a-4523-8108-ac4702fcb412 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:46.428783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bc630-459a-4523-8108-ac4702fcb412 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.430203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bc630-459a-4523-8108-ac4702fcb412 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.447456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:46.448674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 293/585] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:46.449359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bc630-459a-4523-8108-ac4702fcb412 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.514720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d0aa2-e240-4d2a-8a73-7d798e91c075 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.545338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bc630-459a-4523-8108-ac4702fcb412 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.547479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9bdcf-50d4-4e8c-ab5b-0677262be4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.548644 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9bdcf-50d4-4e8c-ab5b-0677262be4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.548901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9bdcf-50d4-4e8c-ab5b-0677262be4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.549176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9bdcf-50d4-4e8c-ab5b-0677262be4f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.555635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9bdcf-50d4-4e8c-ab5b-0677262be4f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.557063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9bdcf-50d4-4e8c-ab5b-0677262be4f3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ddf9bdcf-50d4-4e8c-ab5b-0677262be4f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:46.568582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:46.569852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:46.571051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:46.571051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:46.571051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:46.571051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:46.571051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:46.571051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:46.571051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:46.571051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:46.574671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 293/586] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:46.575635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9bdcf-50d4-4e8c-ab5b-0677262be4f3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.578977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8095d3-4814-4d59-a2f3-8e0eaa923771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.579610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8095d3-4814-4d59-a2f3-8e0eaa923771 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.579882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8095d3-4814-4d59-a2f3-8e0eaa923771 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.580150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8095d3-4814-4d59-a2f3-8e0eaa923771 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.580774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8095d3-4814-4d59-a2f3-8e0eaa923771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:46.581073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8095d3-4814-4d59-a2f3-8e0eaa923771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.582466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8095d3-4814-4d59-a2f3-8e0eaa923771 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.593691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 294/587] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:46.594685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8095d3-4814-4d59-a2f3-8e0eaa923771 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.621849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:46.665085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:46.674083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9bdcf-50d4-4e8c-ab5b-0677262be4f3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.697227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8095d3-4814-4d59-a2f3-8e0eaa923771 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.708720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4896e13-c1f7-40b1-a20a-ec6fee64286e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.709265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4896e13-c1f7-40b1-a20a-ec6fee64286e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.709535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4896e13-c1f7-40b1-a20a-ec6fee64286e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.709805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4896e13-c1f7-40b1-a20a-ec6fee64286e None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.710465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4896e13-c1f7-40b1-a20a-ec6fee64286e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:46.710761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4896e13-c1f7-40b1-a20a-ec6fee64286e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.712118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4896e13-c1f7-40b1-a20a-ec6fee64286e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.722984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 294/588] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:46.723772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4896e13-c1f7-40b1-a20a-ec6fee64286e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.730458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d5286-80b7-4393-ac7f-81698d886032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.731174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d5286-80b7-4393-ac7f-81698d886032 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.731551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d5286-80b7-4393-ac7f-81698d886032 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.731919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d5286-80b7-4393-ac7f-81698d886032 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.732732 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d5286-80b7-4393-ac7f-81698d886032 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:46.733132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d5286-80b7-4393-ac7f-81698d886032 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.734679 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d5286-80b7-4393-ac7f-81698d886032 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.746682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 295/589] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:46.747544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d5286-80b7-4393-ac7f-81698d886032 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.824758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4896e13-c1f7-40b1-a20a-ec6fee64286e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.832152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:46.847845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d5286-80b7-4393-ac7f-81698d886032 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:46.857985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9216dd1-ccc0-43f7-b6a1-ea30c0082583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.858610 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9216dd1-ccc0-43f7-b6a1-ea30c0082583 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.858891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9216dd1-ccc0-43f7-b6a1-ea30c0082583 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.859163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9216dd1-ccc0-43f7-b6a1-ea30c0082583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.870525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9216dd1-ccc0-43f7-b6a1-ea30c0082583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.871963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9216dd1-ccc0-43f7-b6a1-ea30c0082583 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.881061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2fc32-7f20-447f-8113-d57bef9ac242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:46.881666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2fc32-7f20-447f-8113-d57bef9ac242 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:46.881979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2fc32-7f20-447f-8113-d57bef9ac242 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:46.882299 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2fc32-7f20-447f-8113-d57bef9ac242 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:46.883195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2fc32-7f20-447f-8113-d57bef9ac242 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:46.883497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2fc32-7f20-447f-8113-d57bef9ac242 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:46.891509 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2fc32-7f20-447f-8113-d57bef9ac242 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d9216dd1-ccc0-43f7-b6a1-ea30c0082583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:46.891730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:46.892771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:46.893817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:46.893817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:46.893817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:46.893817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 295/590] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:46.893817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9216dd1-ccc0-43f7-b6a1-ea30c0082583 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.896272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:46.902612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 296/591] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:46.903725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2fc32-7f20-447f-8113-d57bef9ac242 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:46.989928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9216dd1-ccc0-43f7-b6a1-ea30c0082583 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:47.002625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2fc32-7f20-447f-8113-d57bef9ac242 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:47.021883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313439-c768-4565-9d9e-f95f8ddb6f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:47.022534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313439-c768-4565-9d9e-f95f8ddb6f95 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:47.022810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313439-c768-4565-9d9e-f95f8ddb6f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:47.023078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47313439-c768-4565-9d9e-f95f8ddb6f95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:47.029453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47313439-c768-4565-9d9e-f95f8ddb6f95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:47.030917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47313439-c768-4565-9d9e-f95f8ddb6f95 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:47.035099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377c973-afde-4f5e-8824-83f90434ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:47.035717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377c973-afde-4f5e-8824-83f90434ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:47.035992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377c973-afde-4f5e-8824-83f90434ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:47.036312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377c973-afde-4f5e-8824-83f90434ccf2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-47313439-c768-4565-9d9e-f95f8ddb6f95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:47.042369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:47.043358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:47.044667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:47.044667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:47.044667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:47.044667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:47.044667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:47.044667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:47.044667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:47.044667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:47.044667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 296/592] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:47.044667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313439-c768-4565-9d9e-f95f8ddb6f95 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:47.045950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377c973-afde-4f5e-8824-83f90434ccf2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:47.047595 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377c973-afde-4f5e-8824-83f90434ccf2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8377c973-afde-4f5e-8824-83f90434ccf2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:47.059519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:47.060669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 297/593] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:47.061874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377c973-afde-4f5e-8824-83f90434ccf2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:47.067601 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:47.108264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:47.145860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47313439-c768-4565-9d9e-f95f8ddb6f95 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:47.163131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377c973-afde-4f5e-8824-83f90434ccf2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:47.178109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625012c-dd97-4e00-8808-c2eb53091d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:47.178735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625012c-dd97-4e00-8808-c2eb53091d3d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:47.179009 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625012c-dd97-4e00-8808-c2eb53091d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:47.179275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625012c-dd97-4e00-8808-c2eb53091d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:47.179991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625012c-dd97-4e00-8808-c2eb53091d3d None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=783ad0cd9721409e8a7114cecef16aab) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:47.180295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625012c-dd97-4e00-8808-c2eb53091d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:47.181723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625012c-dd97-4e00-8808-c2eb53091d3d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:47.198395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 297/594] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:47 2022] GET /identity/v3/role_assignments?user.id=783ad0cd9721409e8a7114cecef16aab&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:47.198395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625012c-dd97-4e00-8808-c2eb53091d3d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:47.198743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0736f51-9c0d-4975-8e88-08814d2aa910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:47.199749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0736f51-9c0d-4975-8e88-08814d2aa910 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:47.200033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0736f51-9c0d-4975-8e88-08814d2aa910 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:47.200299 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0736f51-9c0d-4975-8e88-08814d2aa910 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:47.201433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0736f51-9c0d-4975-8e88-08814d2aa910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:47.201686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0736f51-9c0d-4975-8e88-08814d2aa910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:47.203038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0736f51-9c0d-4975-8e88-08814d2aa910 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:47.222480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 298/595] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:47.222480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0736f51-9c0d-4975-8e88-08814d2aa910 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:47.293296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:47.306440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625012c-dd97-4e00-8808-c2eb53091d3d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:47.338344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:47.341512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0736f51-9c0d-4975-8e88-08814d2aa910 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:47.348307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374542cb-5546-4fc3-a41c-0be8dee4728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:47.349837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374542cb-5546-4fc3-a41c-0be8dee4728c None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:47.349837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374542cb-5546-4fc3-a41c-0be8dee4728c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:47.349837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374542cb-5546-4fc3-a41c-0be8dee4728c None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/c34eb8fcef0543c282b3887dfeb74c15/roles/bc27c203c0234a6194c03f93ee3c50a9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:47.366644 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374542cb-5546-4fc3-a41c-0be8dee4728c None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=c34eb8fcef0543c282b3887dfeb74c15, role_id=bc27c203c0234a6194c03f93ee3c50a9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:47.369089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374542cb-5546-4fc3-a41c-0be8dee4728c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:47.377953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eb523-e9a2-4375-8a18-d78772206981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:47.383128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb523-e9a2-4375-8a18-d78772206981 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:47.383128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb523-e9a2-4375-8a18-d78772206981 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:47.383128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb523-e9a2-4375-8a18-d78772206981 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:47.383128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eb523-e9a2-4375-8a18-d78772206981 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:47.383128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eb523-e9a2-4375-8a18-d78772206981 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:47.383128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eb523-e9a2-4375-8a18-d78772206981 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:47.393660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 299/596] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:47.394471 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eb523-e9a2-4375-8a18-d78772206981 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:47.425708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 298/597] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:24:47 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/c34eb8fcef0543c282b3887dfeb74c15/roles/bc27c203c0234a6194c03f93ee3c50a9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:47.426820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374542cb-5546-4fc3-a41c-0be8dee4728c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:47.518825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eb523-e9a2-4375-8a18-d78772206981 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:47.550045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:47.565504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f3282-eaa2-40a0-bf88-2e0f9ea05bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:47.566129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f3282-eaa2-40a0-bf88-2e0f9ea05bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:47.566435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f3282-eaa2-40a0-bf88-2e0f9ea05bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:47.566687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f3282-eaa2-40a0-bf88-2e0f9ea05bd5 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:47.567422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f3282-eaa2-40a0-bf88-2e0f9ea05bd5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:47.567722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f3282-eaa2-40a0-bf88-2e0f9ea05bd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:47.570376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f3282-eaa2-40a0-bf88-2e0f9ea05bd5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:47.576119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374542cb-5546-4fc3-a41c-0be8dee4728c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:47.598323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 300/598] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Oct 4 05:24:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:47.599610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f3282-eaa2-40a0-bf88-2e0f9ea05bd5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:47.622274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3715f-7131-48fe-9e3c-e6a50f9b2669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:47.623064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3715f-7131-48fe-9e3c-e6a50f9b2669 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:47.623443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3715f-7131-48fe-9e3c-e6a50f9b2669 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:47.623816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3715f-7131-48fe-9e3c-e6a50f9b2669 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:47.633675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3715f-7131-48fe-9e3c-e6a50f9b2669 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:47.635402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3715f-7131-48fe-9e3c-e6a50f9b2669 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-dda3715f-7131-48fe-9e3c-e6a50f9b2669 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:47.648812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:47.649930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:47.651055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:47.651055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:47.651055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:47.651055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:47.651055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:47.651055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:47.651055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:47.651055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:47.652708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 299/599] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:47.653567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3715f-7131-48fe-9e3c-e6a50f9b2669 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:47.702531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:47.726124 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f3282-eaa2-40a0-bf88-2e0f9ea05bd5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:47.764750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0658fe-a31d-4359-b7a2-3b903551a849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:47.765852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0658fe-a31d-4359-b7a2-3b903551a849 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:47.766133 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0658fe-a31d-4359-b7a2-3b903551a849 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:47.766439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0658fe-a31d-4359-b7a2-3b903551a849 None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/fba4f0473f4c4d3bb2d50b09f3de9837/roles/bc27c203c0234a6194c03f93ee3c50a9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:47.779567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3715f-7131-48fe-9e3c-e6a50f9b2669 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:47.786041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0658fe-a31d-4359-b7a2-3b903551a849 None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=fba4f0473f4c4d3bb2d50b09f3de9837, role_id=bc27c203c0234a6194c03f93ee3c50a9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:47.787512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0658fe-a31d-4359-b7a2-3b903551a849 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:47.817056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb6398-432f-4760-b4dc-e24b6ae73df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:47.817639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb6398-432f-4760-b4dc-e24b6ae73df2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:47.817913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb6398-432f-4760-b4dc-e24b6ae73df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:47.818214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb6398-432f-4760-b4dc-e24b6ae73df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:47.829268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb6398-432f-4760-b4dc-e24b6ae73df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:47.835908 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb6398-432f-4760-b4dc-e24b6ae73df2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-cadb6398-432f-4760-b4dc-e24b6ae73df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:47.848606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:47.849560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:47.850515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:47.850515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:47.850515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:47.850515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 300/600] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:47.851196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb6398-432f-4760-b4dc-e24b6ae73df2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:47.864083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 301/601] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:24:47 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/fba4f0473f4c4d3bb2d50b09f3de9837/roles/bc27c203c0234a6194c03f93ee3c50a9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:47.865687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0658fe-a31d-4359-b7a2-3b903551a849 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:48.018529 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:48.032205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:48.052063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb6398-432f-4760-b4dc-e24b6ae73df2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:48.065197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0658fe-a31d-4359-b7a2-3b903551a849 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:48.107115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54b6ea-0905-46f4-8e7a-b3fddd7e3f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:48.108087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54b6ea-0905-46f4-8e7a-b3fddd7e3f54 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:48.108513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54b6ea-0905-46f4-8e7a-b3fddd7e3f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:48.109885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54b6ea-0905-46f4-8e7a-b3fddd7e3f54 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:48.109885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54b6ea-0905-46f4-8e7a-b3fddd7e3f54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:48.110270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54b6ea-0905-46f4-8e7a-b3fddd7e3f54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:48.112425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54b6ea-0905-46f4-8e7a-b3fddd7e3f54 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:48.117285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fe979-a058-4d6f-926a-e8a36de6c4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:48.117907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fe979-a058-4d6f-926a-e8a36de6c4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:48.118859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fe979-a058-4d6f-926a-e8a36de6c4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:48.118859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fe979-a058-4d6f-926a-e8a36de6c4f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:48.119216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fe979-a058-4d6f-926a-e8a36de6c4f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:48.119543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fe979-a058-4d6f-926a-e8a36de6c4f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:48.120866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fe979-a058-4d6f-926a-e8a36de6c4f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:48.127531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 301/602] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:48.133736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54b6ea-0905-46f4-8e7a-b3fddd7e3f54 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:48.138655 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 302/603] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:48.144347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fe979-a058-4d6f-926a-e8a36de6c4f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:48.245915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:48.256755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fe979-a058-4d6f-926a-e8a36de6c4f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:48.312059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a7471-8332-4c44-96a1-bebc13d1abaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:48.312696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a7471-8332-4c44-96a1-bebc13d1abaf None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:48.312974 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a7471-8332-4c44-96a1-bebc13d1abaf None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:48.313241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a7471-8332-4c44-96a1-bebc13d1abaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:48.329617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54b6ea-0905-46f4-8e7a-b3fddd7e3f54 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:48.331605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a7471-8332-4c44-96a1-bebc13d1abaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:48.333049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a7471-8332-4c44-96a1-bebc13d1abaf None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fc6a7471-8332-4c44-96a1-bebc13d1abaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:48.351292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:48.352418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:48.353508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:48.353508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:48.353508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:48.353508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:48.353508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:48.353508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:48.353508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:48.353508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:48.364661 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 303/604] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:48.372833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a7471-8332-4c44-96a1-bebc13d1abaf None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:48.409412 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c3195-17a4-46b6-9f99-858109791d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:48.410084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c3195-17a4-46b6-9f99-858109791d74 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:48.410383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c3195-17a4-46b6-9f99-858109791d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:48.410650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c3195-17a4-46b6-9f99-858109791d74 None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/131afaf047d34e88842fffd0d0b76242/roles/bc27c203c0234a6194c03f93ee3c50a9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:48.441425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c3195-17a4-46b6-9f99-858109791d74 None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=131afaf047d34e88842fffd0d0b76242, role_id=bc27c203c0234a6194c03f93ee3c50a9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:48.442924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c3195-17a4-46b6-9f99-858109791d74 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:48.506208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a7471-8332-4c44-96a1-bebc13d1abaf None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:48.519803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:48.524463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 302/605] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:24:48 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/131afaf047d34e88842fffd0d0b76242/roles/bc27c203c0234a6194c03f93ee3c50a9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:48.565244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a8c78-9dc2-411c-aae3-1974e88ff94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:48.565856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a8c78-9dc2-411c-aae3-1974e88ff94f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:48.566131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a8c78-9dc2-411c-aae3-1974e88ff94f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:48.566422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a8c78-9dc2-411c-aae3-1974e88ff94f None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:48.567112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a8c78-9dc2-411c-aae3-1974e88ff94f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:48.573388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a8c78-9dc2-411c-aae3-1974e88ff94f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:48.573388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a8c78-9dc2-411c-aae3-1974e88ff94f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:48.587967 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 304/606] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:48.593771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c3195-17a4-46b6-9f99-858109791d74 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:48.701852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c3195-17a4-46b6-9f99-858109791d74 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:48.739228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51733c-f2f1-4372-b1a1-558d73dd313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:48.739910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51733c-f2f1-4372-b1a1-558d73dd313f None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:48.740185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51733c-f2f1-4372-b1a1-558d73dd313f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:48.740457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51733c-f2f1-4372-b1a1-558d73dd313f None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/3a0feaf83ee14071806f36d01c36b635/roles/bc27c203c0234a6194c03f93ee3c50a9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:48.756619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51733c-f2f1-4372-b1a1-558d73dd313f None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=3a0feaf83ee14071806f36d01c36b635, role_id=bc27c203c0234a6194c03f93ee3c50a9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:48.758268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51733c-f2f1-4372-b1a1-558d73dd313f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:48.826780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 303/607] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:24:48 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/3a0feaf83ee14071806f36d01c36b635/roles/bc27c203c0234a6194c03f93ee3c50a9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:49.529703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c6033-0dbe-4851-963c-6d5b64a60971 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:49.530580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c6033-0dbe-4851-963c-6d5b64a60971 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:49.531030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c6033-0dbe-4851-963c-6d5b64a60971 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:49.532814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 305/608] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:49.538533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb2513-d748-468e-aa87-24e761ea4788 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:49.538956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb2513-d748-468e-aa87-24e761ea4788 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:49.539266 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb2513-d748-468e-aa87-24e761ea4788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:49.567152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-78fb2513-d748-468e-aa87-24e761ea4788 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:49.568379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-78fb2513-d748-468e-aa87-24e761ea4788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:49.615938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:49.637506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 304/609] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:49.968210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c6033-0dbe-4851-963c-6d5b64a60971 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:49.989024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:50.031421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340b162-f2c8-4716-afa8-5c46353c9393 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:50.031762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340b162-f2c8-4716-afa8-5c46353c9393 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:50.032810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340b162-f2c8-4716-afa8-5c46353c9393 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:50.034772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 305/610] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:24:50.041272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745e96d-4273-4a9b-a764-bfca2464bed9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:50.041988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745e96d-4273-4a9b-a764-bfca2464bed9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:50.042299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745e96d-4273-4a9b-a764-bfca2464bed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:50.065416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4745e96d-4273-4a9b-a764-bfca2464bed9 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:50.066642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4745e96d-4273-4a9b-a764-bfca2464bed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:50.100885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c6033-0dbe-4851-963c-6d5b64a60971 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:50.171471 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921e771-1e02-4573-adee-7ac3e85d65d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:50.172028 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921e771-1e02-4573-adee-7ac3e85d65d3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:50.172302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921e771-1e02-4573-adee-7ac3e85d65d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:50.172571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921e771-1e02-4573-adee-7ac3e85d65d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:50.187896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 306/611] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:50.192411 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0921e771-1e02-4573-adee-7ac3e85d65d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0921e771-1e02-4573-adee-7ac3e85d65d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:50.197548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:50.198494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:50.199953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 306/612] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:50.207177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745e96d-4273-4a9b-a764-bfca2464bed9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:50.308846 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745e96d-4273-4a9b-a764-bfca2464bed9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:50.322492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:50.343874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00855014-7f78-4786-a5ab-c6c82e20268e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:50.344498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00855014-7f78-4786-a5ab-c6c82e20268e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:50.344818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00855014-7f78-4786-a5ab-c6c82e20268e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:50.345707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00855014-7f78-4786-a5ab-c6c82e20268e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:50.346392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00855014-7f78-4786-a5ab-c6c82e20268e None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:50.346791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00855014-7f78-4786-a5ab-c6c82e20268e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:50.348170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00855014-7f78-4786-a5ab-c6c82e20268e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:50.356702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 307/613] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:50.363037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921e771-1e02-4573-adee-7ac3e85d65d3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:50.455488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:50.487657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00855014-7f78-4786-a5ab-c6c82e20268e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:50.495035 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921e771-1e02-4573-adee-7ac3e85d65d3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:50.562006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0763c-c56b-48b1-bb1f-e4480391af31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:50.562813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0763c-c56b-48b1-bb1f-e4480391af31 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:50.563011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0763c-c56b-48b1-bb1f-e4480391af31 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:50.563285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0763c-c56b-48b1-bb1f-e4480391af31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:50.582613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0763c-c56b-48b1-bb1f-e4480391af31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:50.584191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0763c-c56b-48b1-bb1f-e4480391af31 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fda0763c-c56b-48b1-bb1f-e4480391af31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:50.600255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:50.601268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:50.602598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:50.602598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:50.602598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:50.602598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 307/614] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:50.607288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0763c-c56b-48b1-bb1f-e4480391af31 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:50.681787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00855014-7f78-4786-a5ab-c6c82e20268e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:50.728900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97117ab4-7156-4be4-99a1-0b6a885ec4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:50.729873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97117ab4-7156-4be4-99a1-0b6a885ec4bd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:50.730695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97117ab4-7156-4be4-99a1-0b6a885ec4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:50.731155 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97117ab4-7156-4be4-99a1-0b6a885ec4bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:50.744875 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97117ab4-7156-4be4-99a1-0b6a885ec4bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-97117ab4-7156-4be4-99a1-0b6a885ec4bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:50.753635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:50.754592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:50.755432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:50.756070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0763c-c56b-48b1-bb1f-e4480391af31 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:50.758829 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 308/615] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:50.760159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c64995-03b4-42b1-a01d-71d766e8f248 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:50.760601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c64995-03b4-42b1-a01d-71d766e8f248 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:50.761007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c64995-03b4-42b1-a01d-71d766e8f248 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:50.766594 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 309/616] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:50.768752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c64995-03b4-42b1-a01d-71d766e8f248 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:50.792007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:50.799572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d939c-a16f-4708-afcd-5d3ccaea9fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:50.800433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d939c-a16f-4708-afcd-5d3ccaea9fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:50.800907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d939c-a16f-4708-afcd-5d3ccaea9fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:50.801324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d939c-a16f-4708-afcd-5d3ccaea9fa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:50.802120 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d939c-a16f-4708-afcd-5d3ccaea9fa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:50.802638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d939c-a16f-4708-afcd-5d3ccaea9fa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:50.804269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d939c-a16f-4708-afcd-5d3ccaea9fa0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:50.806609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:50.818181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 308/617] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:50.820054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8619ad-de2d-44f6-91d3-891335a7e65e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:50.820543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8619ad-de2d-44f6-91d3-891335a7e65e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:50.821042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8619ad-de2d-44f6-91d3-891335a7e65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:50.840693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1e8619ad-de2d-44f6-91d3-891335a7e65e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:50.845570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1e8619ad-de2d-44f6-91d3-891335a7e65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:50.883330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c64995-03b4-42b1-a01d-71d766e8f248 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:50.918954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cd47e-71cb-45f9-a28d-c1397edb01af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:50.919567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cd47e-71cb-45f9-a28d-c1397edb01af None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:50.919843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cd47e-71cb-45f9-a28d-c1397edb01af None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:50.920112 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cd47e-71cb-45f9-a28d-c1397edb01af None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:50.920830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cd47e-71cb-45f9-a28d-c1397edb01af None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:50.921135 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cd47e-71cb-45f9-a28d-c1397edb01af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:50.922623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cd47e-71cb-45f9-a28d-c1397edb01af None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:50.932660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 310/619] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:50.932945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cd47e-71cb-45f9-a28d-c1397edb01af None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:50.933149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 309/619] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:50.938059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8619ad-de2d-44f6-91d3-891335a7e65e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:51.030654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:51.036267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cd47e-71cb-45f9-a28d-c1397edb01af None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:51.072602 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9adefa-661c-4b98-aa42-390d327290ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:51.080648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9adefa-661c-4b98-aa42-390d327290ce None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.080648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9adefa-661c-4b98-aa42-390d327290ce None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.080648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9adefa-661c-4b98-aa42-390d327290ce None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.081139 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9adefa-661c-4b98-aa42-390d327290ce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:51.082637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9adefa-661c-4b98-aa42-390d327290ce None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9c9adefa-661c-4b98-aa42-390d327290ce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:51.096664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:51.101943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 311/620] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:24:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:51.102940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9adefa-661c-4b98-aa42-390d327290ce None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:51.121800 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8619ad-de2d-44f6-91d3-891335a7e65e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:51.139562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:51.171146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf251f-0bfc-41fd-b37c-adbc4df0f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:51.171797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf251f-0bfc-41fd-b37c-adbc4df0f76e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.172079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf251f-0bfc-41fd-b37c-adbc4df0f76e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.172356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf251f-0bfc-41fd-b37c-adbc4df0f76e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.192440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf251f-0bfc-41fd-b37c-adbc4df0f76e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:51.194166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf251f-0bfc-41fd-b37c-adbc4df0f76e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e0cf251f-0bfc-41fd-b37c-adbc4df0f76e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:51.219435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:51.220688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:51.221796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:51.221796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:51.221796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:51.222004 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 310/621] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:51.226087 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf251f-0bfc-41fd-b37c-adbc4df0f76e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:51.304026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9adefa-661c-4b98-aa42-390d327290ce None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:51.364026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638cf0f-ab6b-4ac9-855c-0c86c9bc083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:51.364734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638cf0f-ab6b-4ac9-855c-0c86c9bc083a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.365087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638cf0f-ab6b-4ac9-855c-0c86c9bc083a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.365430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638cf0f-ab6b-4ac9-855c-0c86c9bc083a None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.366441 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638cf0f-ab6b-4ac9-855c-0c86c9bc083a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:51.366820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638cf0f-ab6b-4ac9-855c-0c86c9bc083a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:51.368754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638cf0f-ab6b-4ac9-855c-0c86c9bc083a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:51.391688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 312/622] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 05:24:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:51.391688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638cf0f-ab6b-4ac9-855c-0c86c9bc083a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:51.406073 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:51.406501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf251f-0bfc-41fd-b37c-adbc4df0f76e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:51.444152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f273e2-99d8-4bc2-b6c7-e5deb3b47fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:51.444782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f273e2-99d8-4bc2-b6c7-e5deb3b47fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.445060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f273e2-99d8-4bc2-b6c7-e5deb3b47fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.445516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f273e2-99d8-4bc2-b6c7-e5deb3b47fd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.446552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f273e2-99d8-4bc2-b6c7-e5deb3b47fd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:51.446879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f273e2-99d8-4bc2-b6c7-e5deb3b47fd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:51.448291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f273e2-99d8-4bc2-b6c7-e5deb3b47fd3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:51.462451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 311/623] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:51.468607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f0f4f-5c74-4a01-bc7b-70946e9491b6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.468607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f0f4f-5c74-4a01-bc7b-70946e9491b6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.468607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f0f4f-5c74-4a01-bc7b-70946e9491b6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.470270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 312/624] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:51.474854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f0f4f-5c74-4a01-bc7b-70946e9491b6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:51.499561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638cf0f-ab6b-4ac9-855c-0c86c9bc083a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:51.506308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:51.534450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ec682-dca1-46a2-8cc0-af3dae085c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:51.535079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec682-dca1-46a2-8cc0-af3dae085c6b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.535353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec682-dca1-46a2-8cc0-af3dae085c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.535620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec682-dca1-46a2-8cc0-af3dae085c6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.545815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ec682-dca1-46a2-8cc0-af3dae085c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-232ec682-dca1-46a2-8cc0-af3dae085c6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:51.551250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:51.552189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 313/625] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:51.553347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ec682-dca1-46a2-8cc0-af3dae085c6b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:51.584559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f0f4f-5c74-4a01-bc7b-70946e9491b6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:51.618454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966eee1-232d-4029-917a-040917025029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:51.619144 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966eee1-232d-4029-917a-040917025029 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.619430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966eee1-232d-4029-917a-040917025029 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.619706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966eee1-232d-4029-917a-040917025029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.622174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:51.630057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966eee1-232d-4029-917a-040917025029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:51.631529 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966eee1-232d-4029-917a-040917025029 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6966eee1-232d-4029-917a-040917025029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:51.647353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:51.648285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:51.649209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:51.649209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:51.649209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:51.649209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 313/626] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:51.649573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffb690-7069-4103-a107-b60206f6cb5f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.649818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffb690-7069-4103-a107-b60206f6cb5f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.650089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffb690-7069-4103-a107-b60206f6cb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.655152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ec682-dca1-46a2-8cc0-af3dae085c6b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:51.668213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-02ffb690-7069-4103-a107-b60206f6cb5f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:51.669406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-02ffb690-7069-4103-a107-b60206f6cb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:51.688734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcceffa8-47b6-4865-b678-16ce3d939bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:51.689410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceffa8-47b6-4865-b678-16ce3d939bdd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.689690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceffa8-47b6-4865-b678-16ce3d939bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.689962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceffa8-47b6-4865-b678-16ce3d939bdd None admin] PATH_INFO: `/v3/users/glance` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.697625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcceffa8-47b6-4865-b678-16ce3d939bdd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:51.699098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcceffa8-47b6-4865-b678-16ce3d939bdd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-fcceffa8-47b6-4865-b678-16ce3d939bdd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:51.710442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:51.711365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:51.712332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 314/627] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:51.712994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a70b-0381-45eb-b54a-073c79cc0a2e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.713063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a70b-0381-45eb-b54a-073c79cc0a2e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.713343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a70b-0381-45eb-b54a-073c79cc0a2e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.714544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 315/628] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:51.715234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1a70b-0381-45eb-b54a-073c79cc0a2e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:51.740523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 314/629] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:51.741893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffb690-7069-4103-a107-b60206f6cb5f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:51.754021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:51.841170 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffb690-7069-4103-a107-b60206f6cb5f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:51.846117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1a70b-0381-45eb-b54a-073c79cc0a2e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:51.876346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:51.878523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdd303-ab3f-4a68-a3d2-bedba2fe930e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:51.879133 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdd303-ab3f-4a68-a3d2-bedba2fe930e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.879415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdd303-ab3f-4a68-a3d2-bedba2fe930e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.879689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdd303-ab3f-4a68-a3d2-bedba2fe930e None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.880344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdd303-ab3f-4a68-a3d2-bedba2fe930e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:51.880670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdd303-ab3f-4a68-a3d2-bedba2fe930e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:51.881817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803be902-abea-49c7-946f-5de8942d7508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:51.882024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdd303-ab3f-4a68-a3d2-bedba2fe930e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:51.882491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803be902-abea-49c7-946f-5de8942d7508 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.882772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803be902-abea-49c7-946f-5de8942d7508 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.883047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803be902-abea-49c7-946f-5de8942d7508 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.883703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803be902-abea-49c7-946f-5de8942d7508 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:51.884007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803be902-abea-49c7-946f-5de8942d7508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:51.885353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803be902-abea-49c7-946f-5de8942d7508 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:51.895644 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 316/630] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:51.896536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803be902-abea-49c7-946f-5de8942d7508 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:51.901603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 315/631] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:51.903228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd987-ef32-4c88-8d7e-0f4bba96a1e2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:51.903707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd987-ef32-4c88-8d7e-0f4bba96a1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:51.904159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd987-ef32-4c88-8d7e-0f4bba96a1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:51.924087 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-af5fd987-ef32-4c88-8d7e-0f4bba96a1e2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:51.925571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-af5fd987-ef32-4c88-8d7e-0f4bba96a1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:51.995565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803be902-abea-49c7-946f-5de8942d7508 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:51.997118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 316/632] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:51.998088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fd987-ef32-4c88-8d7e-0f4bba96a1e2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.026622 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:52.040823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696d5a7-8580-4c25-8570-24b6582eeccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.041701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696d5a7-8580-4c25-8570-24b6582eeccb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.041976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696d5a7-8580-4c25-8570-24b6582eeccb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.042354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696d5a7-8580-4c25-8570-24b6582eeccb None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.043085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696d5a7-8580-4c25-8570-24b6582eeccb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:52.043443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696d5a7-8580-4c25-8570-24b6582eeccb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:52.044907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696d5a7-8580-4c25-8570-24b6582eeccb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:52.068370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 317/633] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 05:24:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:52.068370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696d5a7-8580-4c25-8570-24b6582eeccb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.102223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fd987-ef32-4c88-8d7e-0f4bba96a1e2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.126776 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:52.142705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff79be-1c08-4b1c-ae76-620b338a7c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.143342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff79be-1c08-4b1c-ae76-620b338a7c2b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.143621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff79be-1c08-4b1c-ae76-620b338a7c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.143894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff79be-1c08-4b1c-ae76-620b338a7c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.154970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff79be-1c08-4b1c-ae76-620b338a7c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:52.156475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff79be-1c08-4b1c-ae76-620b338a7c2b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-17ff79be-1c08-4b1c-ae76-620b338a7c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:52.171453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:52.172460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:52.173975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:52.173975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:52.173975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:52.173975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 317/634] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:52.173975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff79be-1c08-4b1c-ae76-620b338a7c2b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.185410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696d5a7-8580-4c25-8570-24b6582eeccb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.221527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aafc22-29e0-4b9a-a525-96d25ef298a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.222297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aafc22-29e0-4b9a-a525-96d25ef298a1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.222577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aafc22-29e0-4b9a-a525-96d25ef298a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.222855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aafc22-29e0-4b9a-a525-96d25ef298a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.225285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:52.230715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aafc22-29e0-4b9a-a525-96d25ef298a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:52.235779 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aafc22-29e0-4b9a-a525-96d25ef298a1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a4aafc22-29e0-4b9a-a525-96d25ef298a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:52.243329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:52.244413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:52.245630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:52.245630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:52.245630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:52.245630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:52.245630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:52.245630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:52.245630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:52.245630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:52.245630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 318/635] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:52.245630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aafc22-29e0-4b9a-a525-96d25ef298a1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.283542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff79be-1c08-4b1c-ae76-620b338a7c2b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.316501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d0c76-e2f3-4b4b-9186-07ce45cc4bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.317136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d0c76-e2f3-4b4b-9186-07ce45cc4bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.317408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d0c76-e2f3-4b4b-9186-07ce45cc4bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.317679 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d0c76-e2f3-4b4b-9186-07ce45cc4bc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.327962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d0c76-e2f3-4b4b-9186-07ce45cc4bc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d76d0c76-e2f3-4b4b-9186-07ce45cc4bc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:52.334484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:52.335453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 318/636] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:52.336294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d0c76-e2f3-4b4b-9186-07ce45cc4bc8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.347836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aafc22-29e0-4b9a-a525-96d25ef298a1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.360901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:52.381512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce76f-7ef7-4345-811e-411ad97d1982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.382117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce76f-7ef7-4345-811e-411ad97d1982 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.382429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce76f-7ef7-4345-811e-411ad97d1982 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.382698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce76f-7ef7-4345-811e-411ad97d1982 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.438039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d0c76-e2f3-4b4b-9186-07ce45cc4bc8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.448471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce76f-7ef7-4345-811e-411ad97d1982 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:52.448859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:52.449914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce76f-7ef7-4345-811e-411ad97d1982 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:52.472735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a9ef8-85b2-4b3c-9c83-5db030fb940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.473331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9ef8-85b2-4b3c-9c83-5db030fb940b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.473608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9ef8-85b2-4b3c-9c83-5db030fb940b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.473882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a9ef8-85b2-4b3c-9c83-5db030fb940b None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.474563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a9ef8-85b2-4b3c-9c83-5db030fb940b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:52.474867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a9ef8-85b2-4b3c-9c83-5db030fb940b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:52.476295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a9ef8-85b2-4b3c-9c83-5db030fb940b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:52.488326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 319/637] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:52.489106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a9ef8-85b2-4b3c-9c83-5db030fb940b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.553224 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 319/638] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:52.554059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce76f-7ef7-4345-811e-411ad97d1982 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.569253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:52.583803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a9ef8-85b2-4b3c-9c83-5db030fb940b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.616730 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae809e8-c31f-4ef4-908e-ba707402f931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.617315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae809e8-c31f-4ef4-908e-ba707402f931 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.617500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae809e8-c31f-4ef4-908e-ba707402f931 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.617770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae809e8-c31f-4ef4-908e-ba707402f931 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.618476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae809e8-c31f-4ef4-908e-ba707402f931 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:52.618775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae809e8-c31f-4ef4-908e-ba707402f931 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:52.620171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae809e8-c31f-4ef4-908e-ba707402f931 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:52.631257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 320/639] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:52.632218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae809e8-c31f-4ef4-908e-ba707402f931 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.654870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce76f-7ef7-4345-811e-411ad97d1982 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.661860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:52.687962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78976fb7-a4da-4e45-b00f-bd958512fa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.688605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78976fb7-a4da-4e45-b00f-bd958512fa0e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.688882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78976fb7-a4da-4e45-b00f-bd958512fa0e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.689413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78976fb7-a4da-4e45-b00f-bd958512fa0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.699963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78976fb7-a4da-4e45-b00f-bd958512fa0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-78976fb7-a4da-4e45-b00f-bd958512fa0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:52.705864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:52.706845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:52.707716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 320/640] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:52.708418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78976fb7-a4da-4e45-b00f-bd958512fa0e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.733644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae809e8-c31f-4ef4-908e-ba707402f931 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.768444 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373fdfb-56f4-49a0-867e-003753a88b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.769050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373fdfb-56f4-49a0-867e-003753a88b82 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.769333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373fdfb-56f4-49a0-867e-003753a88b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.769612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373fdfb-56f4-49a0-867e-003753a88b82 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.770302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373fdfb-56f4-49a0-867e-003753a88b82 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:52.770609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373fdfb-56f4-49a0-867e-003753a88b82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:52.771956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373fdfb-56f4-49a0-867e-003753a88b82 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:52.779420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:52.784476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 321/641] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:52.784476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373fdfb-56f4-49a0-867e-003753a88b82 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.807828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78976fb7-a4da-4e45-b00f-bd958512fa0e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.841845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e060f1-6797-4820-9249-de516d2bc0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.842505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e060f1-6797-4820-9249-de516d2bc0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.842785 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e060f1-6797-4820-9249-de516d2bc0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.843119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e060f1-6797-4820-9249-de516d2bc0d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.849888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e060f1-6797-4820-9249-de516d2bc0d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:52.851329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e060f1-6797-4820-9249-de516d2bc0d8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-11e060f1-6797-4820-9249-de516d2bc0d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:52.862491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:52.863539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:52.864540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 321/642] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 05:24:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:52.865177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e060f1-6797-4820-9249-de516d2bc0d8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.871936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:52.881905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373fdfb-56f4-49a0-867e-003753a88b82 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.916999 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2527932-e365-4fe5-a775-398f5dc94a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:52.917631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2527932-e365-4fe5-a775-398f5dc94a46 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:52.917910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2527932-e365-4fe5-a775-398f5dc94a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:52.918208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2527932-e365-4fe5-a775-398f5dc94a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:52.928207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2527932-e365-4fe5-a775-398f5dc94a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:52.929656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2527932-e365-4fe5-a775-398f5dc94a46 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c2527932-e365-4fe5-a775-398f5dc94a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:52.945460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:52.946575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:52.948076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:52.948076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:52.948076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:52.948076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 322/643] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:52.948076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2527932-e365-4fe5-a775-398f5dc94a46 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:52.970486 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e060f1-6797-4820-9249-de516d2bc0d8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:52.997667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:53.003309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a19efc-f597-4e2b-a21b-ee3d114270df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:53.008334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a19efc-f597-4e2b-a21b-ee3d114270df None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:53.008334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a19efc-f597-4e2b-a21b-ee3d114270df None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:53.008334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a19efc-f597-4e2b-a21b-ee3d114270df None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/783ad0cd9721409e8a7114cecef16aab/roles/bc27c203c0234a6194c03f93ee3c50a9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:53.019622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a19efc-f597-4e2b-a21b-ee3d114270df None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=783ad0cd9721409e8a7114cecef16aab, role_id=bc27c203c0234a6194c03f93ee3c50a9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:53.021058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a19efc-f597-4e2b-a21b-ee3d114270df None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:53.046020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2527932-e365-4fe5-a775-398f5dc94a46 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:53.082444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 322/644] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:24:52 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/783ad0cd9721409e8a7114cecef16aab/roles/bc27c203c0234a6194c03f93ee3c50a9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:24:53.083256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a19efc-f597-4e2b-a21b-ee3d114270df None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:53.095037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4574b-2373-40b3-b3c8-c8d18ff2f200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:53.096038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4574b-2373-40b3-b3c8-c8d18ff2f200 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:53.096306 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4574b-2373-40b3-b3c8-c8d18ff2f200 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:53.096757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4574b-2373-40b3-b3c8-c8d18ff2f200 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:53.097454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4574b-2373-40b3-b3c8-c8d18ff2f200 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:53.097761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4574b-2373-40b3-b3c8-c8d18ff2f200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:53.099445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4574b-2373-40b3-b3c8-c8d18ff2f200 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:53.109727 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 323/645] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:53.111524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4574b-2373-40b3-b3c8-c8d18ff2f200 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:53.122569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:53.199389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a19efc-f597-4e2b-a21b-ee3d114270df None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:53.231083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4574b-2373-40b3-b3c8-c8d18ff2f200 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:53.237530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e629d10-f965-4b77-a153-3dc6665d839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:53.238232 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e629d10-f965-4b77-a153-3dc6665d839d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:53.238513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e629d10-f965-4b77-a153-3dc6665d839d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:53.238785 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e629d10-f965-4b77-a153-3dc6665d839d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:53.249732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e629d10-f965-4b77-a153-3dc6665d839d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:53.251784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e629d10-f965-4b77-a153-3dc6665d839d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8e629d10-f965-4b77-a153-3dc6665d839d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:53.269349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:53.270443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:53.271333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:53.271333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:53.271333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:53.271333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 323/646] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:53.271333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e629d10-f965-4b77-a153-3dc6665d839d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:53.276470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb085775-92c7-4d08-859b-b2791de529c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:53.277088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb085775-92c7-4d08-859b-b2791de529c6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:53.277375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb085775-92c7-4d08-859b-b2791de529c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:53.277627 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb085775-92c7-4d08-859b-b2791de529c6 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:53.278399 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb085775-92c7-4d08-859b-b2791de529c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:53.278703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb085775-92c7-4d08-859b-b2791de529c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:53.280140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb085775-92c7-4d08-859b-b2791de529c6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:53.298883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 324/647] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 05:24:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:53.299800 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb085775-92c7-4d08-859b-b2791de529c6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:53.319552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:53.372559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:53.377053 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e629d10-f965-4b77-a153-3dc6665d839d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:53.412127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55179e7d-8c99-481d-b60f-019100c1fcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:53.412730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55179e7d-8c99-481d-b60f-019100c1fcec None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:53.413003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55179e7d-8c99-481d-b60f-019100c1fcec None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:53.413271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55179e7d-8c99-481d-b60f-019100c1fcec None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:53.413905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55179e7d-8c99-481d-b60f-019100c1fcec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:53.414325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55179e7d-8c99-481d-b60f-019100c1fcec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:53.415597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55179e7d-8c99-481d-b60f-019100c1fcec None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:53.416508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb085775-92c7-4d08-859b-b2791de529c6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:53.428350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 324/648] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:53.429172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55179e7d-8c99-481d-b60f-019100c1fcec None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:53.459282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59413ee-441f-4018-a15f-b5861579a57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:53.460777 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59413ee-441f-4018-a15f-b5861579a57b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:53.461254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59413ee-441f-4018-a15f-b5861579a57b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:53.461717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59413ee-441f-4018-a15f-b5861579a57b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:53.475611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59413ee-441f-4018-a15f-b5861579a57b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:53.477827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59413ee-441f-4018-a15f-b5861579a57b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f59413ee-441f-4018-a15f-b5861579a57b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:53.492895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:53.493896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:53.499831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:53.499831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:53.499831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:53.499831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 325/649] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:53.499831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59413ee-441f-4018-a15f-b5861579a57b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:53.530084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:53.536871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55179e7d-8c99-481d-b60f-019100c1fcec None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:53.573145 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a4945-9d01-4ab1-9c67-5297a0b9d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:53.573754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a4945-9d01-4ab1-9c67-5297a0b9d53f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:53.574028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a4945-9d01-4ab1-9c67-5297a0b9d53f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:53.574327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a4945-9d01-4ab1-9c67-5297a0b9d53f None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:53.574963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a4945-9d01-4ab1-9c67-5297a0b9d53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:53.575262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a4945-9d01-4ab1-9c67-5297a0b9d53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:53.576609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a4945-9d01-4ab1-9c67-5297a0b9d53f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:53.588659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 325/650] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:53.589733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a4945-9d01-4ab1-9c67-5297a0b9d53f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:53.617239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59413ee-441f-4018-a15f-b5861579a57b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:53.633950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:53.651482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76910fec-3628-4251-9fa8-f4f2cc9403a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:53.652183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910fec-3628-4251-9fa8-f4f2cc9403a3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:53.652517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910fec-3628-4251-9fa8-f4f2cc9403a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:53.652790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910fec-3628-4251-9fa8-f4f2cc9403a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:53.693685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a4945-9d01-4ab1-9c67-5297a0b9d53f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:53.724298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76910fec-3628-4251-9fa8-f4f2cc9403a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:53.725833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76910fec-3628-4251-9fa8-f4f2cc9403a3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:53.728541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9beb2-d935-4acf-9cec-003263e48124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:53.729268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9beb2-d935-4acf-9cec-003263e48124 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:53.729615 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9beb2-d935-4acf-9cec-003263e48124 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:53.729955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9beb2-d935-4acf-9cec-003263e48124 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:53.737306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9beb2-d935-4acf-9cec-003263e48124 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:53.738866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9beb2-d935-4acf-9cec-003263e48124 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a9e9beb2-d935-4acf-9cec-003263e48124 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:53.750001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:53.751144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:53.752272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:53.752272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:53.752272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:53.752272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:53.752272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:53.752272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:53.752272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:53.752272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:53.752932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 326/651] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:53.752932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9beb2-d935-4acf-9cec-003263e48124 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:53.771259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:53.837536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 326/652] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:53.838654 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76910fec-3628-4251-9fa8-f4f2cc9403a3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:53.859815 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:53.864973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9beb2-d935-4acf-9cec-003263e48124 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:53.905803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a3cf4-6b62-4e57-8f17-c0f04ccda508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:53.906877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a3cf4-6b62-4e57-8f17-c0f04ccda508 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:53.908271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a3cf4-6b62-4e57-8f17-c0f04ccda508 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:53.908951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a3cf4-6b62-4e57-8f17-c0f04ccda508 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:53.909741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a3cf4-6b62-4e57-8f17-c0f04ccda508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:53.910254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a3cf4-6b62-4e57-8f17-c0f04ccda508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:53.911759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a3cf4-6b62-4e57-8f17-c0f04ccda508 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:53.923200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 327/653] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:53.923824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a3cf4-6b62-4e57-8f17-c0f04ccda508 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:53.964138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76910fec-3628-4251-9fa8-f4f2cc9403a3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.006656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fe17f-bf16-42e8-9162-f8d221fa8a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.006861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:54.007290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fe17f-bf16-42e8-9162-f8d221fa8a2c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.007570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fe17f-bf16-42e8-9162-f8d221fa8a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.007843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fe17f-bf16-42e8-9162-f8d221fa8a2c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:54.016097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fe17f-bf16-42e8-9162-f8d221fa8a2c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:54.018083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fe17f-bf16-42e8-9162-f8d221fa8a2c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-010fe17f-bf16-42e8-9162-f8d221fa8a2c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:54.031196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:54.032183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:54.033639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:54.033639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:54.033639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:54.033639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:54.033639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:54.033639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:54.033639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:54.033639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:54.033639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 05:24:54.033639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:54.034391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a3cf4-6b62-4e57-8f17-c0f04ccda508 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.037683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 327/654] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:54.038545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fe17f-bf16-42e8-9162-f8d221fa8a2c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:54.069429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8643d94-8b11-4f79-8bb7-d430c83b1c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.070034 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8643d94-8b11-4f79-8bb7-d430c83b1c22 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.070348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8643d94-8b11-4f79-8bb7-d430c83b1c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.070620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8643d94-8b11-4f79-8bb7-d430c83b1c22 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:54.071304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8643d94-8b11-4f79-8bb7-d430c83b1c22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:54.071608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8643d94-8b11-4f79-8bb7-d430c83b1c22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:54.073046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8643d94-8b11-4f79-8bb7-d430c83b1c22 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:54.091118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 328/655] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:54.091118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8643d94-8b11-4f79-8bb7-d430c83b1c22 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:54.125376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:54.148848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fe17f-bf16-42e8-9162-f8d221fa8a2c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.186248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9d777-85b6-4d1b-8589-e865aa63048f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.187134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9d777-85b6-4d1b-8589-e865aa63048f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.187414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9d777-85b6-4d1b-8589-e865aa63048f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.187687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9d777-85b6-4d1b-8589-e865aa63048f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:54.195464 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8643d94-8b11-4f79-8bb7-d430c83b1c22 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.199460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9d777-85b6-4d1b-8589-e865aa63048f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:54.200986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9d777-85b6-4d1b-8589-e865aa63048f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3fe9d777-85b6-4d1b-8589-e865aa63048f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:54.217106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:54.218064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:54.219480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:54.219480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:54.219480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:54.219480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 328/656] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:54.219480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9d777-85b6-4d1b-8589-e865aa63048f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:54.239152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff94cef-400f-47b3-841e-c2872ad27065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.240064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff94cef-400f-47b3-841e-c2872ad27065 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.240346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff94cef-400f-47b3-841e-c2872ad27065 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.240616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff94cef-400f-47b3-841e-c2872ad27065 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:54.243131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:54.249826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff94cef-400f-47b3-841e-c2872ad27065 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:54.252625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff94cef-400f-47b3-841e-c2872ad27065 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6ff94cef-400f-47b3-841e-c2872ad27065 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:54.265560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:54.267239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 329/657] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:54.268196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff94cef-400f-47b3-841e-c2872ad27065 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:54.331140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9d777-85b6-4d1b-8589-e865aa63048f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.358581 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:54.368536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0bab-a16f-421f-8818-7347adffccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.369405 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0bab-a16f-421f-8818-7347adffccb7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.369684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0bab-a16f-421f-8818-7347adffccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.372870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0bab-a16f-421f-8818-7347adffccb7 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:54.372870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0bab-a16f-421f-8818-7347adffccb7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:54.372870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0bab-a16f-421f-8818-7347adffccb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:54.372870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0bab-a16f-421f-8818-7347adffccb7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:54.373508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff94cef-400f-47b3-841e-c2872ad27065 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.390948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 329/658] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 05:24:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:54.392447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0bab-a16f-421f-8818-7347adffccb7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:54.413188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e561fdf-e139-4aaf-8fc5-403d050f6efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.413798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e561fdf-e139-4aaf-8fc5-403d050f6efa None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.414074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e561fdf-e139-4aaf-8fc5-403d050f6efa None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.414377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e561fdf-e139-4aaf-8fc5-403d050f6efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:54.415100 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e561fdf-e139-4aaf-8fc5-403d050f6efa None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=c34eb8fcef0543c282b3887dfeb74c15) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:54.415410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e561fdf-e139-4aaf-8fc5-403d050f6efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:54.416826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e561fdf-e139-4aaf-8fc5-403d050f6efa None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:54.434213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 330/659] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:54 2022] GET /identity/v3/role_assignments?user.id=c34eb8fcef0543c282b3887dfeb74c15&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:54.434213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e561fdf-e139-4aaf-8fc5-403d050f6efa None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:54.498080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:54.519056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0bab-a16f-421f-8818-7347adffccb7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.555951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e561fdf-e139-4aaf-8fc5-403d050f6efa None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.564377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad75d0-b0a6-4e0b-80a4-5ca8ee6f8839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.565001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad75d0-b0a6-4e0b-80a4-5ca8ee6f8839 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.565277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad75d0-b0a6-4e0b-80a4-5ca8ee6f8839 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.565542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad75d0-b0a6-4e0b-80a4-5ca8ee6f8839 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:54.566244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad75d0-b0a6-4e0b-80a4-5ca8ee6f8839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:54.566548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad75d0-b0a6-4e0b-80a4-5ca8ee6f8839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:54.568065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad75d0-b0a6-4e0b-80a4-5ca8ee6f8839 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:54.582611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 330/660] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:54.583483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad75d0-b0a6-4e0b-80a4-5ca8ee6f8839 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:54.603391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392a2b9-f684-4958-a466-9272f4e11456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.604546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392a2b9-f684-4958-a466-9272f4e11456 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.605420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392a2b9-f684-4958-a466-9272f4e11456 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.605883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392a2b9-f684-4958-a466-9272f4e11456 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:54.606833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392a2b9-f684-4958-a466-9272f4e11456 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:54.607329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392a2b9-f684-4958-a466-9272f4e11456 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:54.609640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392a2b9-f684-4958-a466-9272f4e11456 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:54.630695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 331/661] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Oct 4 05:24:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:54.631943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392a2b9-f684-4958-a466-9272f4e11456 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:54.636656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:54.718247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad75d0-b0a6-4e0b-80a4-5ca8ee6f8839 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.760347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392a2b9-f684-4958-a466-9272f4e11456 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.766009 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fede829-48f9-4451-a5f3-b7365687ce4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.766895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fede829-48f9-4451-a5f3-b7365687ce4b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.767323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fede829-48f9-4451-a5f3-b7365687ce4b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.767737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fede829-48f9-4451-a5f3-b7365687ce4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:54.780654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:54.809204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3e0a1-e6f1-4951-8bb5-579c2ae5f4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.809966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3e0a1-e6f1-4951-8bb5-579c2ae5f4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.810368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3e0a1-e6f1-4951-8bb5-579c2ae5f4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.811008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3e0a1-e6f1-4951-8bb5-579c2ae5f4d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:54.821155 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3e0a1-e6f1-4951-8bb5-579c2ae5f4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:54.822992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3e0a1-e6f1-4951-8bb5-579c2ae5f4d2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e4b3e0a1-e6f1-4951-8bb5-579c2ae5f4d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:54.839450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:54.840590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:54.841730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:54.841730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:54.841730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:54.841730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:54.841730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:54.841730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:54.841730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:54.841730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:54.842573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 332/662] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:54.846775 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3e0a1-e6f1-4951-8bb5-579c2ae5f4d2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:54.854763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fede829-48f9-4451-a5f3-b7365687ce4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:54.856530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fede829-48f9-4451-a5f3-b7365687ce4b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:54.925889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:54.961807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3e0a1-e6f1-4951-8bb5-579c2ae5f4d2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:54.996609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 331/663] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:54.997414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113ce79-9a3f-41e1-bf41-7946dc2d751c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:54.997558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fede829-48f9-4451-a5f3-b7365687ce4b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:54.998577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113ce79-9a3f-41e1-bf41-7946dc2d751c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:54.999071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113ce79-9a3f-41e1-bf41-7946dc2d751c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:54.999557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113ce79-9a3f-41e1-bf41-7946dc2d751c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.047644 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:55.072947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113ce79-9a3f-41e1-bf41-7946dc2d751c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:55.074876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113ce79-9a3f-41e1-bf41-7946dc2d751c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:55.126115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fede829-48f9-4451-a5f3-b7365687ce4b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:55.174290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc82971-4601-4d1f-a727-72312c539af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:55.175553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc82971-4601-4d1f-a727-72312c539af9 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:55.176034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc82971-4601-4d1f-a727-72312c539af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:55.176582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc82971-4601-4d1f-a727-72312c539af9 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.177481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc82971-4601-4d1f-a727-72312c539af9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:55.178041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc82971-4601-4d1f-a727-72312c539af9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:55.179729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc82971-4601-4d1f-a727-72312c539af9 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:55.191065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 333/664] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:24:55.191963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113ce79-9a3f-41e1-bf41-7946dc2d751c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:55.193639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 332/665] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:55.199654 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc82971-4601-4d1f-a727-72312c539af9 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:55.209027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:55.274961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:55.298847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113ce79-9a3f-41e1-bf41-7946dc2d751c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:55.305666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc82971-4601-4d1f-a727-72312c539af9 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:55.338447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3805a-1f20-45a5-aa1e-520f0e9effe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:55.339080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3805a-1f20-45a5-aa1e-520f0e9effe2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:55.339362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3805a-1f20-45a5-aa1e-520f0e9effe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:55.339637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3805a-1f20-45a5-aa1e-520f0e9effe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.341538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9a2dd-d0f9-4abe-9a2b-33efcdf56334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:55.342203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a2dd-d0f9-4abe-9a2b-33efcdf56334 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:55.342481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a2dd-d0f9-4abe-9a2b-33efcdf56334 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:55.342753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a2dd-d0f9-4abe-9a2b-33efcdf56334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.350538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3805a-1f20-45a5-aa1e-520f0e9effe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:55.352143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3805a-1f20-45a5-aa1e-520f0e9effe2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:55.354121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a2dd-d0f9-4abe-9a2b-33efcdf56334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:55.355582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a2dd-d0f9-4abe-9a2b-33efcdf56334 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b9d3805a-1f20-45a5-aa1e-520f0e9effe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:55.367251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:55.368226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:55.374028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:55.374028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:55.374028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:55.374028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 334/666] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:55.374028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3805a-1f20-45a5-aa1e-520f0e9effe2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-16a9a2dd-d0f9-4abe-9a2b-33efcdf56334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:55.374362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:55.375193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:55.379073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:55.379073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:55.379073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:55.379073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 333/667] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:55.379073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bb2fa-be1f-4e7c-acf4-9dcbd9768a14 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:55.379073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bb2fa-be1f-4e7c-acf4-9dcbd9768a14 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:55.379073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bb2fa-be1f-4e7c-acf4-9dcbd9768a14 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.379073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 334/668] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:55.379073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bb2fa-be1f-4e7c-acf4-9dcbd9768a14 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:55.442512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:55.487131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3805a-1f20-45a5-aa1e-520f0e9effe2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:55.492325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bb2fa-be1f-4e7c-acf4-9dcbd9768a14 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:55.519366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:55.527206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad6874-9319-43e9-af49-7116bdab9e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:55.532895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb839e5-2ade-48d3-b160-b40e2e4ae60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:55.533052 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6874-9319-43e9-af49-7116bdab9e38 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:55.533052 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6874-9319-43e9-af49-7116bdab9e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:55.533213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb839e5-2ade-48d3-b160-b40e2e4ae60a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:55.533306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad6874-9319-43e9-af49-7116bdab9e38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.533389 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb839e5-2ade-48d3-b160-b40e2e4ae60a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:55.533389 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb839e5-2ade-48d3-b160-b40e2e4ae60a None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.533534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad6874-9319-43e9-af49-7116bdab9e38 None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=fba4f0473f4c4d3bb2d50b09f3de9837) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:55.533534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad6874-9319-43e9-af49-7116bdab9e38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:55.533688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb839e5-2ade-48d3-b160-b40e2e4ae60a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:55.533688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb839e5-2ade-48d3-b160-b40e2e4ae60a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:55.533841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad6874-9319-43e9-af49-7116bdab9e38 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:55.533922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb839e5-2ade-48d3-b160-b40e2e4ae60a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:55.538592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 335/669] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:55 2022] GET /identity/v3/role_assignments?user.id=fba4f0473f4c4d3bb2d50b09f3de9837&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:55.539240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad6874-9319-43e9-af49-7116bdab9e38 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:55.662981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad6874-9319-43e9-af49-7116bdab9e38 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:55.696085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 335/670] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:55.696085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15068fc-8f76-4955-a736-3f1408325da1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:55.696085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15068fc-8f76-4955-a736-3f1408325da1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:55.696085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15068fc-8f76-4955-a736-3f1408325da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.709502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5133a7-1034-4425-b81c-9016fa02ed4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:55.710561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5133a7-1034-4425-b81c-9016fa02ed4a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:55.711089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5133a7-1034-4425-b81c-9016fa02ed4a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:55.711558 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5133a7-1034-4425-b81c-9016fa02ed4a None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.712432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5133a7-1034-4425-b81c-9016fa02ed4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:55.712930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5133a7-1034-4425-b81c-9016fa02ed4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:55.714415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e15068fc-8f76-4955-a736-3f1408325da1 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:55.714926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5133a7-1034-4425-b81c-9016fa02ed4a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:55.715634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e15068fc-8f76-4955-a736-3f1408325da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:55.730057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 336/671] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:55.731332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5133a7-1034-4425-b81c-9016fa02ed4a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:55.793800 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:55.807423 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 336/672] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:55.808568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15068fc-8f76-4955-a736-3f1408325da1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:55.856499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5133a7-1034-4425-b81c-9016fa02ed4a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:55.868837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:55.903633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62298c50-50c8-4deb-bcaf-850e8e60af0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:55.904514 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62298c50-50c8-4deb-bcaf-850e8e60af0b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:55.904868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62298c50-50c8-4deb-bcaf-850e8e60af0b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:55.905148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62298c50-50c8-4deb-bcaf-850e8e60af0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.915850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62298c50-50c8-4deb-bcaf-850e8e60af0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:55.915850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62298c50-50c8-4deb-bcaf-850e8e60af0b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-62298c50-50c8-4deb-bcaf-850e8e60af0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:55.928550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:55.929628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:55.930531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:55.930531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:55.930531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:55.930531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:55.930531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:55.930531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:55.930531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:55.930531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:55.930531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 337/673] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:55.935586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15068fc-8f76-4955-a736-3f1408325da1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:55.937103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62298c50-50c8-4deb-bcaf-850e8e60af0b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:55.981758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252a7c1-4c11-4cfa-8ba6-ab13178461a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:55.982806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252a7c1-4c11-4cfa-8ba6-ab13178461a6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:55.983289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252a7c1-4c11-4cfa-8ba6-ab13178461a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:55.983761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252a7c1-4c11-4cfa-8ba6-ab13178461a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:55.991693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252a7c1-4c11-4cfa-8ba6-ab13178461a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:55.994196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252a7c1-4c11-4cfa-8ba6-ab13178461a6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-6252a7c1-4c11-4cfa-8ba6-ab13178461a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:56.007880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:56.008784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:56.009665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:56.009665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:56.009665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:56.009665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:56.009665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:56.009665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:56.009665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:56.009665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:56.013069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 337/674] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:56.023316 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252a7c1-4c11-4cfa-8ba6-ab13178461a6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:56.091976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62298c50-50c8-4deb-bcaf-850e8e60af0b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:56.118484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:56.140973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1411b-5ce2-40c4-b71d-6786fac612a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:56.141605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1411b-5ce2-40c4-b71d-6786fac612a6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:56.141882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1411b-5ce2-40c4-b71d-6786fac612a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:56.142169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1411b-5ce2-40c4-b71d-6786fac612a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:56.142872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1411b-5ce2-40c4-b71d-6786fac612a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:56.143230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1411b-5ce2-40c4-b71d-6786fac612a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:56.144843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1411b-5ce2-40c4-b71d-6786fac612a6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:56.162238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 338/675] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:56.163483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1411b-5ce2-40c4-b71d-6786fac612a6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:56.174857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252a7c1-4c11-4cfa-8ba6-ab13178461a6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:56.184717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:56.210262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffefb3-3351-4995-85c8-d488a3842860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:56.210875 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffefb3-3351-4995-85c8-d488a3842860 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:56.211151 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffefb3-3351-4995-85c8-d488a3842860 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:56.211496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdffefb3-3351-4995-85c8-d488a3842860 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:56.212186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffefb3-3351-4995-85c8-d488a3842860 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:56.212521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffefb3-3351-4995-85c8-d488a3842860 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:56.213929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdffefb3-3351-4995-85c8-d488a3842860 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:56.226810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 338/676] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:56.227634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffefb3-3351-4995-85c8-d488a3842860 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:56.284735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1411b-5ce2-40c4-b71d-6786fac612a6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:56.329540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff385c5-959d-4475-9ba7-397534f1a4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:56.330249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff385c5-959d-4475-9ba7-397534f1a4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:56.330534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff385c5-959d-4475-9ba7-397534f1a4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:56.330804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff385c5-959d-4475-9ba7-397534f1a4d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:56.344185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff385c5-959d-4475-9ba7-397534f1a4d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-cff385c5-959d-4475-9ba7-397534f1a4d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:56.351588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:56.352720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 339/677] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:24:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:56.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff385c5-959d-4475-9ba7-397534f1a4d7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:56.363294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdffefb3-3351-4995-85c8-d488a3842860 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:56.398111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:56.407739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b8997-b728-4fd0-b2ab-d3eaef11321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:56.408612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b8997-b728-4fd0-b2ab-d3eaef11321f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:56.409061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b8997-b728-4fd0-b2ab-d3eaef11321f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:56.409510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b8997-b728-4fd0-b2ab-d3eaef11321f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:56.410868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b8997-b728-4fd0-b2ab-d3eaef11321f None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=131afaf047d34e88842fffd0d0b76242) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:56.413545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b8997-b728-4fd0-b2ab-d3eaef11321f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:56.413545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b8997-b728-4fd0-b2ab-d3eaef11321f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:56.428337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 339/678] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:56 2022] GET /identity/v3/role_assignments?user.id=131afaf047d34e88842fffd0d0b76242&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:56.428337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b8997-b728-4fd0-b2ab-d3eaef11321f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:56.500518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:56.537187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff385c5-959d-4475-9ba7-397534f1a4d7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:56.613760 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4557b-5219-4cb4-956b-d693133da624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:56.614727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4557b-5219-4cb4-956b-d693133da624 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:56.617495 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4557b-5219-4cb4-956b-d693133da624 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:56.617811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4557b-5219-4cb4-956b-d693133da624 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:56.618603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4557b-5219-4cb4-956b-d693133da624 None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=3a0feaf83ee14071806f36d01c36b635) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:56.618918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4557b-5219-4cb4-956b-d693133da624 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:56.621606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4557b-5219-4cb4-956b-d693133da624 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:56.633641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 340/679] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:56 2022] GET /identity/v3/role_assignments?user.id=3a0feaf83ee14071806f36d01c36b635&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:56.650232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b8997-b728-4fd0-b2ab-d3eaef11321f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:56.703703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796854f9-08c0-483d-a123-b6d3008c911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:56.704428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796854f9-08c0-483d-a123-b6d3008c911d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:56.704649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796854f9-08c0-483d-a123-b6d3008c911d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:56.704917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796854f9-08c0-483d-a123-b6d3008c911d None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:56.705709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796854f9-08c0-483d-a123-b6d3008c911d None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:56.706025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796854f9-08c0-483d-a123-b6d3008c911d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:56.707638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796854f9-08c0-483d-a123-b6d3008c911d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:56.719259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 340/680] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:24:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:56.725844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4557b-5219-4cb4-956b-d693133da624 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:56.841371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:56.853028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4557b-5219-4cb4-956b-d693133da624 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:56.867104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abe7bd-bedd-4d4c-8f8a-5f78055e75a9 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:56.867404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abe7bd-bedd-4d4c-8f8a-5f78055e75a9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:56.867666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abe7bd-bedd-4d4c-8f8a-5f78055e75a9 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:56.869012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 341/681] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:56.874325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d74f5-0b69-4e16-baee-6f1fdf3086eb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:56.874660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d74f5-0b69-4e16-baee-6f1fdf3086eb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:56.874880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d74f5-0b69-4e16-baee-6f1fdf3086eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:56.911521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bac823-94d2-4a17-89f1-3ed38df61f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:56.912172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bac823-94d2-4a17-89f1-3ed38df61f25 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:56.912449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bac823-94d2-4a17-89f1-3ed38df61f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:56.912725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bac823-94d2-4a17-89f1-3ed38df61f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:56.914237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b47d74f5-0b69-4e16-baee-6f1fdf3086eb None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:56.915658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b47d74f5-0b69-4e16-baee-6f1fdf3086eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:56.933592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bac823-94d2-4a17-89f1-3ed38df61f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:56.936669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bac823-94d2-4a17-89f1-3ed38df61f25 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e2bac823-94d2-4a17-89f1-3ed38df61f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:56.953479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:56.955096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:56.956137 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:56.956137 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:56.956137 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:56.959365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 341/682] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:56.970285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bac823-94d2-4a17-89f1-3ed38df61f25 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:57.009152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 342/683] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:57.088694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bac823-94d2-4a17-89f1-3ed38df61f25 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:57.124512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6a991-8c8a-4fa4-9589-58dc6ca98d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:57.125126 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6a991-8c8a-4fa4-9589-58dc6ca98d8f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:57.125406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6a991-8c8a-4fa4-9589-58dc6ca98d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:57.125681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6a991-8c8a-4fa4-9589-58dc6ca98d8f None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:57.126401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b6a991-8c8a-4fa4-9589-58dc6ca98d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:57.126714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b6a991-8c8a-4fa4-9589-58dc6ca98d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:57.128077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b6a991-8c8a-4fa4-9589-58dc6ca98d8f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:57.144309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 342/684] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:57.146975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d74f5-0b69-4e16-baee-6f1fdf3086eb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:57.177692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:57.285668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d74f5-0b69-4e16-baee-6f1fdf3086eb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:57.318035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6a991-8c8a-4fa4-9589-58dc6ca98d8f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:57.332691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e063f-32e6-4622-bd40-b86b5cb2b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:57.333337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e063f-32e6-4622-bd40-b86b5cb2b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:57.333616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e063f-32e6-4622-bd40-b86b5cb2b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:57.333891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e063f-32e6-4622-bd40-b86b5cb2b7b8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:57.341711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e063f-32e6-4622-bd40-b86b5cb2b7b8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:57.343171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e063f-32e6-4622-bd40-b86b5cb2b7b8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:57.344253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2d3e063f-32e6-4622-bd40-b86b5cb2b7b8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:57.356096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:57.360913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:57.361866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:57.361866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:57.361866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:57.361866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:57.361866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:57.361866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:57.361866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:57.361866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:57.361866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:24:57.361866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:57.362443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 343/685] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:24:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:57.364034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e063f-32e6-4622-bd40-b86b5cb2b7b8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:57.491631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:57.507820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6a991-8c8a-4fa4-9589-58dc6ca98d8f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:57.524584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e063f-32e6-4622-bd40-b86b5cb2b7b8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:57.575412 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4f69f-5b2d-4762-b8e4-acd3a89d25a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:57.575969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4f69f-5b2d-4762-b8e4-acd3a89d25a2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:57.576243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4f69f-5b2d-4762-b8e4-acd3a89d25a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:57.576513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4f69f-5b2d-4762-b8e4-acd3a89d25a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:57.588047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4f69f-5b2d-4762-b8e4-acd3a89d25a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:57.589491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4f69f-5b2d-4762-b8e4-acd3a89d25a2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:57.590180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed410f3b-0a9a-46d6-a784-b70cd9adb3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:57.590801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed410f3b-0a9a-46d6-a784-b70cd9adb3aa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:57.591080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed410f3b-0a9a-46d6-a784-b70cd9adb3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:57.591350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed410f3b-0a9a-46d6-a784-b70cd9adb3aa None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:57.592079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed410f3b-0a9a-46d6-a784-b70cd9adb3aa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:57.592385 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed410f3b-0a9a-46d6-a784-b70cd9adb3aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:57.593771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed410f3b-0a9a-46d6-a784-b70cd9adb3aa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-81b4f69f-5b2d-4762-b8e4-acd3a89d25a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:57.604309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:57.605246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:57.606169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:57.606169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:57.606169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:57.606169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 343/686] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:57.614771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4f69f-5b2d-4762-b8e4-acd3a89d25a2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:57.626972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 344/687] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 05:24:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:57.632367 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed410f3b-0a9a-46d6-a784-b70cd9adb3aa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:57.734534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:57.800269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4f69f-5b2d-4762-b8e4-acd3a89d25a2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:57.801603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed410f3b-0a9a-46d6-a784-b70cd9adb3aa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:57.847390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6bd92-4965-418b-8e14-e6768b05f5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:57.848286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6bd92-4965-418b-8e14-e6768b05f5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:57.848707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6bd92-4965-418b-8e14-e6768b05f5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:57.849121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6bd92-4965-418b-8e14-e6768b05f5e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:57.869897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:57.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e8d21-8616-41b2-924d-484d40965ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:57.880250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e8d21-8616-41b2-924d-484d40965ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:57.880718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e8d21-8616-41b2-924d-484d40965ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:57.881192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e8d21-8616-41b2-924d-484d40965ca2 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:57.882167 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e8d21-8616-41b2-924d-484d40965ca2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:57.883459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e8d21-8616-41b2-924d-484d40965ca2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:57.886777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e8d21-8616-41b2-924d-484d40965ca2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:57.912196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 344/688] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:57.918306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e8d21-8616-41b2-924d-484d40965ca2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:57.980850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6bd92-4965-418b-8e14-e6768b05f5e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:57.982312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6bd92-4965-418b-8e14-e6768b05f5e0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:58.112995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e8d21-8616-41b2-924d-484d40965ca2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:58.154528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:58.166534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9a285-99ed-40e9-9fbf-2ff622825e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:58.167301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9a285-99ed-40e9-9fbf-2ff622825e1b None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:58.167593 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9a285-99ed-40e9-9fbf-2ff622825e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:58.167875 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9a285-99ed-40e9-9fbf-2ff622825e1b None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:58.168531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9a285-99ed-40e9-9fbf-2ff622825e1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:58.170172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9a285-99ed-40e9-9fbf-2ff622825e1b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:58.203838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 345/689] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:24:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 05:24:58.206305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6bd92-4965-418b-8e14-e6768b05f5e0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:58.252160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:58.286363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 345/690] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:24:57 2022] POST /identity/v3/users => generated 254 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:24:58.287755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2ef75-9ac5-45c1-ad2c-c0d8ae069ce6 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:58.288036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2ef75-9ac5-45c1-ad2c-c0d8ae069ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:58.288314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2ef75-9ac5-45c1-ad2c-c0d8ae069ce6 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:58.289737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 346/691] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:58.296739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf80c18-a03f-4668-91dc-3cdad6d42d92 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:58.297368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf80c18-a03f-4668-91dc-3cdad6d42d92 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:58.298482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf80c18-a03f-4668-91dc-3cdad6d42d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:58.325750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fcf80c18-a03f-4668-91dc-3cdad6d42d92 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:58.327632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fcf80c18-a03f-4668-91dc-3cdad6d42d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:58.331950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6bd92-4965-418b-8e14-e6768b05f5e0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:58.411854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 347/692] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:24:58.430939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510d212-e0de-4630-9a1f-175c7d08eec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:58.431599 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510d212-e0de-4630-9a1f-175c7d08eec6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:58.431878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510d212-e0de-4630-9a1f-175c7d08eec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:58.432153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510d212-e0de-4630-9a1f-175c7d08eec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:58.454985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510d212-e0de-4630-9a1f-175c7d08eec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:58.456442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510d212-e0de-4630-9a1f-175c7d08eec6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c510d212-e0de-4630-9a1f-175c7d08eec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:58.481941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:58.482988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:58.483883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:58.483883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:58.483883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:58.486543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 346/693] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:58.489154 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf80c18-a03f-4668-91dc-3cdad6d42d92 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:58.629562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf80c18-a03f-4668-91dc-3cdad6d42d92 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:58.637515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:58.666564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041bfdb-50e9-4888-8c8b-8fb1b4637618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:58.667282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041bfdb-50e9-4888-8c8b-8fb1b4637618 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:58.667951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041bfdb-50e9-4888-8c8b-8fb1b4637618 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:58.668225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041bfdb-50e9-4888-8c8b-8fb1b4637618 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:58.668979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041bfdb-50e9-4888-8c8b-8fb1b4637618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:58.669344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041bfdb-50e9-4888-8c8b-8fb1b4637618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:58.670880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041bfdb-50e9-4888-8c8b-8fb1b4637618 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:58.688558 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 348/694] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:58.690001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510d212-e0de-4630-9a1f-175c7d08eec6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:58.771311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041bfdb-50e9-4888-8c8b-8fb1b4637618 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:58.795481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:58.802705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510d212-e0de-4630-9a1f-175c7d08eec6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:58.842044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8273f-84cf-4f59-90f4-aed3a6199d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:58.842714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8273f-84cf-4f59-90f4-aed3a6199d57 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:58.842995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8273f-84cf-4f59-90f4-aed3a6199d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:58.843270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8273f-84cf-4f59-90f4-aed3a6199d57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:58.850902 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8273f-84cf-4f59-90f4-aed3a6199d57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:58.852353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8273f-84cf-4f59-90f4-aed3a6199d57 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b0b8273f-84cf-4f59-90f4-aed3a6199d57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:24:58.864866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:58.865786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:58.866701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:24:58.866701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:24:58.866701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:24:58.866701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:24:58.866701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:24:58.866701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:24:58.866701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:24:58.866701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:58.870368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 347/695] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:58.872218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8273f-84cf-4f59-90f4-aed3a6199d57 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:58.890473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041bfdb-50e9-4888-8c8b-8fb1b4637618 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:58.934116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494467ea-b390-49c3-98b9-27316c500d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:58.935161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494467ea-b390-49c3-98b9-27316c500d53 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:58.935668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494467ea-b390-49c3-98b9-27316c500d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:58.936135 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494467ea-b390-49c3-98b9-27316c500d53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:58.950552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494467ea-b390-49c3-98b9-27316c500d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-494467ea-b390-49c3-98b9-27316c500d53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:24:58.957705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:58.958942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:24:58.959772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:58.962354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 349/696] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:24:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:58.968639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494467ea-b390-49c3-98b9-27316c500d53 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:58.991218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8273f-84cf-4f59-90f4-aed3a6199d57 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:58.998710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:59.029250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a51aa-eafd-4a1d-8930-9138df427431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:59.029875 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a51aa-eafd-4a1d-8930-9138df427431 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.030204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a51aa-eafd-4a1d-8930-9138df427431 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.030437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a51aa-eafd-4a1d-8930-9138df427431 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.031135 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a51aa-eafd-4a1d-8930-9138df427431 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:59.031441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a51aa-eafd-4a1d-8930-9138df427431 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:59.033010 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a51aa-eafd-4a1d-8930-9138df427431 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:59.050158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 348/697] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:24:59.051975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a51aa-eafd-4a1d-8930-9138df427431 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:59.069279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:59.085426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494467ea-b390-49c3-98b9-27316c500d53 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:59.120135 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ee23c-f262-46a5-8f59-70df98d9ae53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:59.120751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ee23c-f262-46a5-8f59-70df98d9ae53 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.121031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ee23c-f262-46a5-8f59-70df98d9ae53 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.121305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ee23c-f262-46a5-8f59-70df98d9ae53 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.121975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ee23c-f262-46a5-8f59-70df98d9ae53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:59.122335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ee23c-f262-46a5-8f59-70df98d9ae53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:59.123764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ee23c-f262-46a5-8f59-70df98d9ae53 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:59.133026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 350/698] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:24:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:59.138517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6839f3-995d-4c6b-a638-15454b8e2fb2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.138517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6839f3-995d-4c6b-a638-15454b8e2fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.138517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6839f3-995d-4c6b-a638-15454b8e2fb2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.138517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 351/699] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:59.139408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6839f3-995d-4c6b-a638-15454b8e2fb2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:59.192732 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a51aa-eafd-4a1d-8930-9138df427431 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:59.242518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:59.243489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebccae-a81d-4795-9548-72af92dc8ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:59.244613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebccae-a81d-4795-9548-72af92dc8ebe None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.245105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebccae-a81d-4795-9548-72af92dc8ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.245725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebccae-a81d-4795-9548-72af92dc8ebe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.246756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebccae-a81d-4795-9548-72af92dc8ebe None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=783ad0cd9721409e8a7114cecef16aab) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:59.247044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6839f3-995d-4c6b-a638-15454b8e2fb2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:59.247467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebccae-a81d-4795-9548-72af92dc8ebe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:59.250189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebccae-a81d-4795-9548-72af92dc8ebe None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:59.258881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 349/700] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:24:59 2022] GET /identity/v3/role_assignments?user.id=783ad0cd9721409e8a7114cecef16aab&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:59.260553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444f296-2aff-4558-abbe-fb911331dd3c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.261031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444f296-2aff-4558-abbe-fb911331dd3c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.261588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444f296-2aff-4558-abbe-fb911331dd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.283307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4444f296-2aff-4558-abbe-fb911331dd3c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:59.284846 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4444f296-2aff-4558-abbe-fb911331dd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:59.335500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c1334-cfb6-4e31-8c2f-b084b87265a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:59.336197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c1334-cfb6-4e31-8c2f-b084b87265a9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.336477 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c1334-cfb6-4e31-8c2f-b084b87265a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.336752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c1334-cfb6-4e31-8c2f-b084b87265a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.348234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c1334-cfb6-4e31-8c2f-b084b87265a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:59.349762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c1334-cfb6-4e31-8c2f-b084b87265a9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:59.370287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 350/701] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:59.371709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351e916-e04d-4337-af1e-301fb2a47056 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.372005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351e916-e04d-4337-af1e-301fb2a47056 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.372347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351e916-e04d-4337-af1e-301fb2a47056 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-085c1334-cfb6-4e31-8c2f-b084b87265a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:24:59.373891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:59.374850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:24:59.376323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:24:59.376323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:24:59.376323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:59.378611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 352/703] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:24:59.378729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 351/703] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:24:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:24:59.383283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927524-6736-412f-8893-65ff01282548 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.383283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927524-6736-412f-8893-65ff01282548 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.383283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927524-6736-412f-8893-65ff01282548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.384765 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c1334-cfb6-4e31-8c2f-b084b87265a9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:59.400544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-91927524-6736-412f-8893-65ff01282548 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:24:59.401552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-91927524-6736-412f-8893-65ff01282548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:24:59.430532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:59.478608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 352/704] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:24:59.512067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c1334-cfb6-4e31-8c2f-b084b87265a9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:59.546910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88c40f-8f4f-4461-b3a8-0a41b42cd816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:59.547569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88c40f-8f4f-4461-b3a8-0a41b42cd816 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.548333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88c40f-8f4f-4461-b3a8-0a41b42cd816 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.548603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88c40f-8f4f-4461-b3a8-0a41b42cd816 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.549236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88c40f-8f4f-4461-b3a8-0a41b42cd816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:59.549531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88c40f-8f4f-4461-b3a8-0a41b42cd816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:59.550910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88c40f-8f4f-4461-b3a8-0a41b42cd816 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:59.559938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:59.564738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 353/705] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:59.570532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91927524-6736-412f-8893-65ff01282548 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:59.674820 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91927524-6736-412f-8893-65ff01282548 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:59.698389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88c40f-8f4f-4461-b3a8-0a41b42cd816 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:59.723863 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dd921-c840-45dc-994f-c9d2c2a92328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:59.724500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dd921-c840-45dc-994f-c9d2c2a92328 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.724777 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dd921-c840-45dc-994f-c9d2c2a92328 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.725063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dd921-c840-45dc-994f-c9d2c2a92328 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.736097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dd921-c840-45dc-994f-c9d2c2a92328 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:59.737864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dd921-c840-45dc-994f-c9d2c2a92328 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:59.740679 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-588dd921-c840-45dc-994f-c9d2c2a92328 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:24:59.756287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:59.757364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 353/706] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 05:24:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:59.766519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dd921-c840-45dc-994f-c9d2c2a92328 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:59.839175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88c40f-8f4f-4461-b3a8-0a41b42cd816 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:59.879571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08db73f-1184-4d4a-bd72-217f9db4facf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:59.880773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08db73f-1184-4d4a-bd72-217f9db4facf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.880873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dd921-c840-45dc-994f-c9d2c2a92328 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:24:59.881305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08db73f-1184-4d4a-bd72-217f9db4facf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.881305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08db73f-1184-4d4a-bd72-217f9db4facf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.881519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08db73f-1184-4d4a-bd72-217f9db4facf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:59.883247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08db73f-1184-4d4a-bd72-217f9db4facf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d08db73f-1184-4d4a-bd72-217f9db4facf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:24:59.906302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:24:59.911361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:24:59.912218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 354/707] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Oct 4 05:24:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:24:59.913021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08db73f-1184-4d4a-bd72-217f9db4facf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:59.923918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca737ec3-e623-48af-8aeb-15b1c9a71657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:24:59.925070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca737ec3-e623-48af-8aeb-15b1c9a71657 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:24:59.925563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca737ec3-e623-48af-8aeb-15b1c9a71657 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:24:59.925955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca737ec3-e623-48af-8aeb-15b1c9a71657 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:24:59.926772 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca737ec3-e623-48af-8aeb-15b1c9a71657 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:24:59.927128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca737ec3-e623-48af-8aeb-15b1c9a71657 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:24:59.929309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca737ec3-e623-48af-8aeb-15b1c9a71657 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:24:59.952899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 354/708] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 05:24:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:24:59.954362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca737ec3-e623-48af-8aeb-15b1c9a71657 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:24:59.981590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:00.020110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:00.024905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08db73f-1184-4d4a-bd72-217f9db4facf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:00.064195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69191b99-0cd1-4690-bab3-9e8fc644caee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:00.065180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69191b99-0cd1-4690-bab3-9e8fc644caee None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:00.065625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69191b99-0cd1-4690-bab3-9e8fc644caee None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:00.066043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69191b99-0cd1-4690-bab3-9e8fc644caee None admin] PATH_INFO: `/v3/services/network` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:00.066808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69191b99-0cd1-4690-bab3-9e8fc644caee None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:00.068612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69191b99-0cd1-4690-bab3-9e8fc644caee None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:00.075304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca737ec3-e623-48af-8aeb-15b1c9a71657 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-69191b99-0cd1-4690-bab3-9e8fc644caee None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:00.079294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:00.080324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 05:25:00.081179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:00.083849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 355/709] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:24:59 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:00.084836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69191b99-0cd1-4690-bab3-9e8fc644caee None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:00.114558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d62d1-4a98-4b30-9b18-8403bde6a365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:00.115169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d62d1-4a98-4b30-9b18-8403bde6a365 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:00.115538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d62d1-4a98-4b30-9b18-8403bde6a365 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:00.115818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d62d1-4a98-4b30-9b18-8403bde6a365 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:00.116548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d62d1-4a98-4b30-9b18-8403bde6a365 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:00.116853 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d62d1-4a98-4b30-9b18-8403bde6a365 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:00.118367 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d62d1-4a98-4b30-9b18-8403bde6a365 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:00.123535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 355/710] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 05:24:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:00.124443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d62d1-4a98-4b30-9b18-8403bde6a365 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:00.209083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69191b99-0cd1-4690-bab3-9e8fc644caee None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:00.232613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d62d1-4a98-4b30-9b18-8403bde6a365 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:00.247398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:00.255871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:00.261173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08aa3d-2437-4d45-a5d2-dea81bc32df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:00.261798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08aa3d-2437-4d45-a5d2-dea81bc32df7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:00.262077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08aa3d-2437-4d45-a5d2-dea81bc32df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:00.262569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08aa3d-2437-4d45-a5d2-dea81bc32df7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:00.274047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa0cf8-b86e-44fe-bbcc-7e74821b62cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:00.274643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa0cf8-b86e-44fe-bbcc-7e74821b62cc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:00.274925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa0cf8-b86e-44fe-bbcc-7e74821b62cc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:00.275200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa0cf8-b86e-44fe-bbcc-7e74821b62cc None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:00.276440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa0cf8-b86e-44fe-bbcc-7e74821b62cc None admin] RBAC: Adding query filter params (name=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:00.276991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa0cf8-b86e-44fe-bbcc-7e74821b62cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:00.279029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa0cf8-b86e-44fe-bbcc-7e74821b62cc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:00.284623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 356/711] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:00.285622 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa0cf8-b86e-44fe-bbcc-7e74821b62cc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:00.411171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa0cf8-b86e-44fe-bbcc-7e74821b62cc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:00.453448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb265d33-0c9b-4989-8177-94f56537e1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:00.455334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb265d33-0c9b-4989-8177-94f56537e1bd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:00.455794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb265d33-0c9b-4989-8177-94f56537e1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:00.456244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb265d33-0c9b-4989-8177-94f56537e1bd None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:00.457071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb265d33-0c9b-4989-8177-94f56537e1bd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:00.458091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb265d33-0c9b-4989-8177-94f56537e1bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:00.459937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb265d33-0c9b-4989-8177-94f56537e1bd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:00.467483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 357/712] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 05:25:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:00.468481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb265d33-0c9b-4989-8177-94f56537e1bd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:00.486087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08aa3d-2437-4d45-a5d2-dea81bc32df7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:00.487694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08aa3d-2437-4d45-a5d2-dea81bc32df7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:00.528940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:00.617739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb265d33-0c9b-4989-8177-94f56537e1bd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:00.699614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330ed2a-b7d2-4760-a598-6973e59a7561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:00.700566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330ed2a-b7d2-4760-a598-6973e59a7561 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:00.701213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330ed2a-b7d2-4760-a598-6973e59a7561 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:00.702384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330ed2a-b7d2-4760-a598-6973e59a7561 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:00.703216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330ed2a-b7d2-4760-a598-6973e59a7561 None admin] RBAC: Adding query filter params (type=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:00.703699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330ed2a-b7d2-4760-a598-6973e59a7561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:00.705735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330ed2a-b7d2-4760-a598-6973e59a7561 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:00.712129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 358/713] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:00.714253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 356/714] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:25:00.719254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330ed2a-b7d2-4760-a598-6973e59a7561 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:00.863443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330ed2a-b7d2-4760-a598-6973e59a7561 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:00.890524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:00.932268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfa794-a163-4be9-b0aa-6eb88e6be68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:00.932923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfa794-a163-4be9-b0aa-6eb88e6be68f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:00.933204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfa794-a163-4be9-b0aa-6eb88e6be68f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:00.933481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfa794-a163-4be9-b0aa-6eb88e6be68f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:00.945606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfa794-a163-4be9-b0aa-6eb88e6be68f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:00.947098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfa794-a163-4be9-b0aa-6eb88e6be68f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-46bfa794-a163-4be9-b0aa-6eb88e6be68f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:00.962700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:00.963687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:00.964618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:00.964618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:00.964618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:00.964618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 359/715] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:00.971402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08aa3d-2437-4d45-a5d2-dea81bc32df7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:01.002632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:01.082902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08aa3d-2437-4d45-a5d2-dea81bc32df7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:01.131736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6d180-6693-4b1a-9c3e-6b2b47da4303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:01.132586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6d180-6693-4b1a-9c3e-6b2b47da4303 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:01.133004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6d180-6693-4b1a-9c3e-6b2b47da4303 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:01.133446 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6d180-6693-4b1a-9c3e-6b2b47da4303 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:01.134253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6d180-6693-4b1a-9c3e-6b2b47da4303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:01.134710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6d180-6693-4b1a-9c3e-6b2b47da4303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:01.136213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6d180-6693-4b1a-9c3e-6b2b47da4303 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:01.152670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 357/716] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:01.158303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfa794-a163-4be9-b0aa-6eb88e6be68f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:01.218459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db50d1-f807-481a-a9ff-e4a9a7e951e8 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:01.218980 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db50d1-f807-481a-a9ff-e4a9a7e951e8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:01.219530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db50d1-f807-481a-a9ff-e4a9a7e951e8 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:01.221237 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 358/717] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:01.236721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224157-8da6-4b99-bb5d-7317867e3a5e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:01.237212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224157-8da6-4b99-bb5d-7317867e3a5e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:01.237718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224157-8da6-4b99-bb5d-7317867e3a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:01.271358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3b224157-8da6-4b99-bb5d-7317867e3a5e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:01.272913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3b224157-8da6-4b99-bb5d-7317867e3a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:01.274953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfa794-a163-4be9-b0aa-6eb88e6be68f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:01.310124 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b865a48-d1e3-4a56-b00d-21f8dca92df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:01.310891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b865a48-d1e3-4a56-b00d-21f8dca92df4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:01.311173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b865a48-d1e3-4a56-b00d-21f8dca92df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:01.311446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b865a48-d1e3-4a56-b00d-21f8dca92df4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:01.319741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b865a48-d1e3-4a56-b00d-21f8dca92df4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:01.321196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b865a48-d1e3-4a56-b00d-21f8dca92df4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2b865a48-d1e3-4a56-b00d-21f8dca92df4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:01.333665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:01.334618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:01.335531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:01.335531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:01.335531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:01.335531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:01.335531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:01.335531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:01.335531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:01.335531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:01.335531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 360/718] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:01.347236 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b865a48-d1e3-4a56-b00d-21f8dca92df4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:01.351150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 359/719] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:25:01.373899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:01.453498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b865a48-d1e3-4a56-b00d-21f8dca92df4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:01.489548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8a0f5-4e9a-41d4-861d-b146303c1aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:01.490180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8a0f5-4e9a-41d4-861d-b146303c1aee None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:01.490510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8a0f5-4e9a-41d4-861d-b146303c1aee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:01.490762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8a0f5-4e9a-41d4-861d-b146303c1aee None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:01.491457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8a0f5-4e9a-41d4-861d-b146303c1aee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:01.491765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8a0f5-4e9a-41d4-861d-b146303c1aee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:01.493184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8a0f5-4e9a-41d4-861d-b146303c1aee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:01.505889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 361/720] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:01.512129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b224157-8da6-4b99-bb5d-7317867e3a5e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:01.542824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:01.653136 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b224157-8da6-4b99-bb5d-7317867e3a5e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:01.663309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8a0f5-4e9a-41d4-861d-b146303c1aee None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:01.691138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25748ece-d342-478a-9e68-be31ae1f6208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:01.691757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748ece-d342-478a-9e68-be31ae1f6208 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:01.692057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748ece-d342-478a-9e68-be31ae1f6208 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:01.692813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748ece-d342-478a-9e68-be31ae1f6208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:01.693561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25748ece-d342-478a-9e68-be31ae1f6208 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=fba4f0473f4c4d3bb2d50b09f3de9837) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:01.694130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25748ece-d342-478a-9e68-be31ae1f6208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:01.695761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25748ece-d342-478a-9e68-be31ae1f6208 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:01.702298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 360/721] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:01 2022] GET /identity/v3/role_assignments?user.id=fba4f0473f4c4d3bb2d50b09f3de9837&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:01.808552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:01.829495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8a0f5-4e9a-41d4-861d-b146303c1aee None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:01.877770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc6932-6098-44a9-b061-6e56f0fb6351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:01.878446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc6932-6098-44a9-b061-6e56f0fb6351 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:01.878725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc6932-6098-44a9-b061-6e56f0fb6351 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:01.878995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc6932-6098-44a9-b061-6e56f0fb6351 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:01.894711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc6932-6098-44a9-b061-6e56f0fb6351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-dfdc6932-6098-44a9-b061-6e56f0fb6351 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:01.899702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:01.900643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:01.901492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 362/722] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:25:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:01.908263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25748ece-d342-478a-9e68-be31ae1f6208 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:02.029362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:02.048394 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25748ece-d342-478a-9e68-be31ae1f6208 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:02.093433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788a81-5d1d-4479-9d9e-752113768411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:02.094056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788a81-5d1d-4479-9d9e-752113768411 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:02.094396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788a81-5d1d-4479-9d9e-752113768411 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:02.094670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788a81-5d1d-4479-9d9e-752113768411 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:02.095335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788a81-5d1d-4479-9d9e-752113768411 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:02.095638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788a81-5d1d-4479-9d9e-752113768411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:02.097060 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788a81-5d1d-4479-9d9e-752113768411 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:02.107217 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 361/723] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:25:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:02.112944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc6932-6098-44a9-b061-6e56f0fb6351 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:02.265412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc6932-6098-44a9-b061-6e56f0fb6351 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:02.287473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7246f2-c362-42bb-87a8-e2ecd81b7aab None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:02.287784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7246f2-c362-42bb-87a8-e2ecd81b7aab None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:02.288015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7246f2-c362-42bb-87a8-e2ecd81b7aab None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:02.289605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 362/724] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:02.294550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c777-4077-4d40-9298-2b5d51168ddf None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:02.294832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c777-4077-4d40-9298-2b5d51168ddf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:02.295109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042c777-4077-4d40-9298-2b5d51168ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:02.317255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458fa1-1062-4193-b2ff-9b9d8bf934e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:02.317892 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458fa1-1062-4193-b2ff-9b9d8bf934e2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:02.318187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458fa1-1062-4193-b2ff-9b9d8bf934e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:02.318453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05458fa1-1062-4193-b2ff-9b9d8bf934e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:02.320027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7042c777-4077-4d40-9298-2b5d51168ddf None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:02.321214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7042c777-4077-4d40-9298-2b5d51168ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:02.338716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05458fa1-1062-4193-b2ff-9b9d8bf934e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:02.340170 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05458fa1-1062-4193-b2ff-9b9d8bf934e2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:02.347282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-05458fa1-1062-4193-b2ff-9b9d8bf934e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:02.364277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:02.365259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:02.366189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:02.366189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:02.366189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:02.366189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 363/725] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:02.372086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458fa1-1062-4193-b2ff-9b9d8bf934e2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:02.452194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 363/726] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 05:25:02.530008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05458fa1-1062-4193-b2ff-9b9d8bf934e2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:02.582813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5101e-3c66-44f5-ade1-f728cce6785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:02.583424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5101e-3c66-44f5-ade1-f728cce6785e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:02.583703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5101e-3c66-44f5-ade1-f728cce6785e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:02.583973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5101e-3c66-44f5-ade1-f728cce6785e None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:02.584603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5101e-3c66-44f5-ade1-f728cce6785e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:02.584903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5101e-3c66-44f5-ade1-f728cce6785e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:02.586266 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5101e-3c66-44f5-ade1-f728cce6785e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:02.598670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 364/727] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:02.604204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042c777-4077-4d40-9298-2b5d51168ddf None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:02.701635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:02.733727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042c777-4077-4d40-9298-2b5d51168ddf None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:02.779746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539f81-8bc6-4dc6-86ab-af84f8cbb8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:02.780386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539f81-8bc6-4dc6-86ab-af84f8cbb8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:02.780662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539f81-8bc6-4dc6-86ab-af84f8cbb8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:02.780938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539f81-8bc6-4dc6-86ab-af84f8cbb8d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:02.795369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22539f81-8bc6-4dc6-86ab-af84f8cbb8d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:02.796812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22539f81-8bc6-4dc6-86ab-af84f8cbb8d1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-22539f81-8bc6-4dc6-86ab-af84f8cbb8d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:02.810674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:02.811676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:02.812914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 364/728] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 05:25:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:02.819882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5101e-3c66-44f5-ade1-f728cce6785e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:02.891768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:02.947745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5101e-3c66-44f5-ade1-f728cce6785e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:02.995069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9851fe8-8b1e-4156-a99d-6081caaa48aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:02.995678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9851fe8-8b1e-4156-a99d-6081caaa48aa None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:02.995956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9851fe8-8b1e-4156-a99d-6081caaa48aa None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:02.996230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9851fe8-8b1e-4156-a99d-6081caaa48aa None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:02.997242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9851fe8-8b1e-4156-a99d-6081caaa48aa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:02.997554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9851fe8-8b1e-4156-a99d-6081caaa48aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:02.998968 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9851fe8-8b1e-4156-a99d-6081caaa48aa None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:03.013853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539f81-8bc6-4dc6-86ab-af84f8cbb8d1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:03.039670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 365/729] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 05:25:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:03.046695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9851fe8-8b1e-4156-a99d-6081caaa48aa None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:03.168608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539f81-8bc6-4dc6-86ab-af84f8cbb8d1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:03.206990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67592cb-78c1-4567-9357-99a2fb5d7d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:03.207627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67592cb-78c1-4567-9357-99a2fb5d7d79 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:03.207904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67592cb-78c1-4567-9357-99a2fb5d7d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:03.208181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67592cb-78c1-4567-9357-99a2fb5d7d79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:03.210015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:03.222179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67592cb-78c1-4567-9357-99a2fb5d7d79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c67592cb-78c1-4567-9357-99a2fb5d7d79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:03.227888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:03.228970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:03.230440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 365/730] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:03.235370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67592cb-78c1-4567-9357-99a2fb5d7d79 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:03.276801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9851fe8-8b1e-4156-a99d-6081caaa48aa None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:03.296787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:03.326722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2b783-a8fe-4526-a46a-3defc4f07877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:03.327617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2b783-a8fe-4526-a46a-3defc4f07877 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:03.328042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2b783-a8fe-4526-a46a-3defc4f07877 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:03.328459 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2b783-a8fe-4526-a46a-3defc4f07877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:03.410196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2b783-a8fe-4526-a46a-3defc4f07877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:03.411664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2b783-a8fe-4526-a46a-3defc4f07877 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:03.419756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67592cb-78c1-4567-9357-99a2fb5d7d79 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:03.456561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5d36f-9b67-42c4-9721-c23ee39f749e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:03.457174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5d36f-9b67-42c4-9721-c23ee39f749e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:03.457450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5d36f-9b67-42c4-9721-c23ee39f749e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:03.457749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5d36f-9b67-42c4-9721-c23ee39f749e None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:03.458440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5d36f-9b67-42c4-9721-c23ee39f749e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:03.458733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5d36f-9b67-42c4-9721-c23ee39f749e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:03.460100 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5d36f-9b67-42c4-9721-c23ee39f749e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:03.474720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 366/731] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:25:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:03.476867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35300b17-a3e6-4328-a5e6-103f7f48eafa None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:03.477356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35300b17-a3e6-4328-a5e6-103f7f48eafa None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:03.477830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35300b17-a3e6-4328-a5e6-103f7f48eafa None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:03.484481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 367/732] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:03.484481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a99b8d-a0cf-48e6-95f6-8a8cfe60e23f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:03.484481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a99b8d-a0cf-48e6-95f6-8a8cfe60e23f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:03.484481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a99b8d-a0cf-48e6-95f6-8a8cfe60e23f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:03.484481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 368/733] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:03.485604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a99b8d-a0cf-48e6-95f6-8a8cfe60e23f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:03.532767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 366/734] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 05:25:03.532767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10bc0a-3c53-4563-9acf-016b8fdd9846 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:03.532767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10bc0a-3c53-4563-9acf-016b8fdd9846 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:03.532767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10bc0a-3c53-4563-9acf-016b8fdd9846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:03.549362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:03.555181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0a10bc0a-3c53-4563-9acf-016b8fdd9846 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:03.556330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0a10bc0a-3c53-4563-9acf-016b8fdd9846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:03.566657 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:03.594980 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a99b8d-a0cf-48e6-95f6-8a8cfe60e23f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:03.629652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bab5e-c72c-48af-8829-5e3cf0c2991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:03.630309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bab5e-c72c-48af-8829-5e3cf0c2991a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:03.630584 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bab5e-c72c-48af-8829-5e3cf0c2991a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:03.630851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bab5e-c72c-48af-8829-5e3cf0c2991a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:03.634470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 367/735] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:25:03.635983 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc23a0-7a09-4025-b5dc-3bbbfefd19ac None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:03.636495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc23a0-7a09-4025-b5dc-3bbbfefd19ac None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:03.636985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc23a0-7a09-4025-b5dc-3bbbfefd19ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:03.641503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bab5e-c72c-48af-8829-5e3cf0c2991a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:03.642927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bab5e-c72c-48af-8829-5e3cf0c2991a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:03.658868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3ebc23a0-7a09-4025-b5dc-3bbbfefd19ac None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a12bab5e-c72c-48af-8829-5e3cf0c2991a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:03.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:03.660099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:03.660991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:03.660991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:03.660991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:03.662842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3ebc23a0-7a09-4025-b5dc-3bbbfefd19ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:03.664812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 369/736] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:03.665844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bab5e-c72c-48af-8829-5e3cf0c2991a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:03.754596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 368/737] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 05:25:03.755404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc23a0-7a09-4025-b5dc-3bbbfefd19ac None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:03.792565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bab5e-c72c-48af-8829-5e3cf0c2991a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:03.803958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:03.832904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37d9ad-fe53-4696-9e95-067b2e8f8b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:03.833924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37d9ad-fe53-4696-9e95-067b2e8f8b39 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:03.834425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37d9ad-fe53-4696-9e95-067b2e8f8b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:03.835211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37d9ad-fe53-4696-9e95-067b2e8f8b39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:03.848339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37d9ad-fe53-4696-9e95-067b2e8f8b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:03.850133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37d9ad-fe53-4696-9e95-067b2e8f8b39 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-de37d9ad-fe53-4696-9e95-067b2e8f8b39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:03.867288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:03.868269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:03.869162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:03.869162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:03.869162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:03.871410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 370/738] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:03.877293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37d9ad-fe53-4696-9e95-067b2e8f8b39 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:03.955741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:03.989855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc23a0-7a09-4025-b5dc-3bbbfefd19ac None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:04.005291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37d9ad-fe53-4696-9e95-067b2e8f8b39 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:04.034972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e5f41-c965-4ceb-8530-989558d2189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:04.035585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e5f41-c965-4ceb-8530-989558d2189b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:04.035868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e5f41-c965-4ceb-8530-989558d2189b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:04.036140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e5f41-c965-4ceb-8530-989558d2189b None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:04.036782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e5f41-c965-4ceb-8530-989558d2189b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:04.037085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e5f41-c965-4ceb-8530-989558d2189b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:04.038466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e5f41-c965-4ceb-8530-989558d2189b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:04.045105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32cc61-f0b3-46e3-9223-28e3bd6a86e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:04.045958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32cc61-f0b3-46e3-9223-28e3bd6a86e7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:04.046457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32cc61-f0b3-46e3-9223-28e3bd6a86e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:04.046944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32cc61-f0b3-46e3-9223-28e3bd6a86e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:04.048347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32cc61-f0b3-46e3-9223-28e3bd6a86e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:04.048843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32cc61-f0b3-46e3-9223-28e3bd6a86e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:04.050425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32cc61-f0b3-46e3-9223-28e3bd6a86e7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:04.056032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 369/739] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:04.056032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e5f41-c965-4ceb-8530-989558d2189b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:04.068539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 371/740] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:04.069876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32cc61-f0b3-46e3-9223-28e3bd6a86e7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:04.091959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:04.159913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e5f41-c965-4ceb-8530-989558d2189b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:04.171811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32cc61-f0b3-46e3-9223-28e3bd6a86e7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:04.194177 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573f68-036a-43eb-9c28-72d3596ff842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:04.194788 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573f68-036a-43eb-9c28-72d3596ff842 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:04.195072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573f68-036a-43eb-9c28-72d3596ff842 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:04.195348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9573f68-036a-43eb-9c28-72d3596ff842 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:04.195996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9573f68-036a-43eb-9c28-72d3596ff842 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:04.197358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9573f68-036a-43eb-9c28-72d3596ff842 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:04.200743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:04.210171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f43392-862f-4dbc-b8a1-4a072625c639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:04.210816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f43392-862f-4dbc-b8a1-4a072625c639 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:04.211091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f43392-862f-4dbc-b8a1-4a072625c639 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:04.215865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f43392-862f-4dbc-b8a1-4a072625c639 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:04.220095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f43392-862f-4dbc-b8a1-4a072625c639 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:04.221596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f43392-862f-4dbc-b8a1-4a072625c639 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-08f43392-862f-4dbc-b8a1-4a072625c639 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:04.235950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:04.237169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 372/741] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:04.244663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f43392-862f-4dbc-b8a1-4a072625c639 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:04.322310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 370/742] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:25:04 2022] POST /identity/v3/services => generated 240 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:04.322310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573f68-036a-43eb-9c28-72d3596ff842 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:04.330134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:04.346787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f43392-862f-4dbc-b8a1-4a072625c639 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:04.396397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce73b74-49ed-4730-9d0e-7d6a1e62698a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:04.397405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce73b74-49ed-4730-9d0e-7d6a1e62698a None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:04.397879 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce73b74-49ed-4730-9d0e-7d6a1e62698a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:04.398411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce73b74-49ed-4730-9d0e-7d6a1e62698a None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:04.399427 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce73b74-49ed-4730-9d0e-7d6a1e62698a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:04.400899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce73b74-49ed-4730-9d0e-7d6a1e62698a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:04.448765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9573f68-036a-43eb-9c28-72d3596ff842 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:04.466854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 373/743] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:25:04 2022] POST /identity/v3/services => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:04.467781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce73b74-49ed-4730-9d0e-7d6a1e62698a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:04.519331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce49d5-676a-49c0-8b98-a09ce5f55309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:04.520098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce49d5-676a-49c0-8b98-a09ce5f55309 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:04.520388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce49d5-676a-49c0-8b98-a09ce5f55309 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:04.520708 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce49d5-676a-49c0-8b98-a09ce5f55309 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:04.535228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce49d5-676a-49c0-8b98-a09ce5f55309 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:04.536871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce49d5-676a-49c0-8b98-a09ce5f55309 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-edce49d5-676a-49c0-8b98-a09ce5f55309 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:04.550428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:04.551424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:04.552917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:04.552917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:04.552917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:04.552917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:04.552917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:04.552917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:04.552917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:04.552917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:04.552917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 371/744] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:04.552917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89ea29-59fd-4d66-8072-5a65e9c6be74 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:04.553495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89ea29-59fd-4d66-8072-5a65e9c6be74 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:04.553495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89ea29-59fd-4d66-8072-5a65e9c6be74 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:04.555225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 372/745] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:04.558175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89ea29-59fd-4d66-8072-5a65e9c6be74 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:04.609680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce73b74-49ed-4730-9d0e-7d6a1e62698a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:04.634465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:04.652236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a39a69-b8e2-4e65-b06d-034fedcefcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:04.653504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a39a69-b8e2-4e65-b06d-034fedcefcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:04.653504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a39a69-b8e2-4e65-b06d-034fedcefcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:04.653504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a39a69-b8e2-4e65-b06d-034fedcefcf9 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:04.654285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a39a69-b8e2-4e65-b06d-034fedcefcf9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:04.654598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a39a69-b8e2-4e65-b06d-034fedcefcf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:04.656122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a39a69-b8e2-4e65-b06d-034fedcefcf9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:04.660335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:04.678096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 374/746] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 05:25:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:04.679413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0adb7-6c71-4449-b112-bdbd6270c309 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:04.682676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0adb7-6c71-4449-b112-bdbd6270c309 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:04.682971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0adb7-6c71-4449-b112-bdbd6270c309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:04.705796 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-47a0adb7-6c71-4449-b112-bdbd6270c309 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:04.706452 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89ea29-59fd-4d66-8072-5a65e9c6be74 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:04.707030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-47a0adb7-6c71-4449-b112-bdbd6270c309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:04.763618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6cd7a-9ae9-4096-a6cd-8072f0960de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:04.764487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6cd7a-9ae9-4096-a6cd-8072f0960de6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:04.764947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6cd7a-9ae9-4096-a6cd-8072f0960de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:04.765898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6cd7a-9ae9-4096-a6cd-8072f0960de6 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:04.767670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6cd7a-9ae9-4096-a6cd-8072f0960de6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:04.768165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6cd7a-9ae9-4096-a6cd-8072f0960de6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:04.770524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6cd7a-9ae9-4096-a6cd-8072f0960de6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:04.781227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 375/747] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:04.790161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0adb7-6c71-4449-b112-bdbd6270c309 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:04.790303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 373/748] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:04.795773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6cd7a-9ae9-4096-a6cd-8072f0960de6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:04.944286 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0adb7-6c71-4449-b112-bdbd6270c309 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:04.996292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6cd7a-9ae9-4096-a6cd-8072f0960de6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:04.999170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267dcc-63a5-4716-be00-98f131be5728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:04.999788 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267dcc-63a5-4716-be00-98f131be5728 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:05.000070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267dcc-63a5-4716-be00-98f131be5728 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:05.000344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267dcc-63a5-4716-be00-98f131be5728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:05.010495 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:05.019917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:05.084289 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd267dcc-63a5-4716-be00-98f131be5728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:05.085763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd267dcc-63a5-4716-be00-98f131be5728 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:05.089540 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87cee4-bb13-4adf-abe4-49c167b947d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:05.090168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87cee4-bb13-4adf-abe4-49c167b947d7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:05.090458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87cee4-bb13-4adf-abe4-49c167b947d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:05.090734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87cee4-bb13-4adf-abe4-49c167b947d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:05.091467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87cee4-bb13-4adf-abe4-49c167b947d7 None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=5cf157961e3d45d5ad861f9b4f51ab50) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:05.091781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87cee4-bb13-4adf-abe4-49c167b947d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:05.094054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87cee4-bb13-4adf-abe4-49c167b947d7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:05.105415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 374/749] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:04 2022] GET /identity/v3/role_assignments?user.id=5cf157961e3d45d5ad861f9b4f51ab50&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:05.111764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87cee4-bb13-4adf-abe4-49c167b947d7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:05.295000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87cee4-bb13-4adf-abe4-49c167b947d7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:05.309554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 376/750] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:25:05.322176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267dcc-63a5-4716-be00-98f131be5728 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:05.351820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:05.356568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d2348-1045-499a-b41a-8f19d59459cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:05.358302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d2348-1045-499a-b41a-8f19d59459cb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:05.361206 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d2348-1045-499a-b41a-8f19d59459cb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:05.361494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d2348-1045-499a-b41a-8f19d59459cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:05.387524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d2348-1045-499a-b41a-8f19d59459cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d15d2348-1045-499a-b41a-8f19d59459cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:05.393378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:05.394330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:05.395155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 375/751] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:05.403086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d2348-1045-499a-b41a-8f19d59459cb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:05.453946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267dcc-63a5-4716-be00-98f131be5728 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:05.491430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb96b47-9fe8-4f10-a5e9-6174a63e03c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:05.492423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96b47-9fe8-4f10-a5e9-6174a63e03c6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:05.492900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96b47-9fe8-4f10-a5e9-6174a63e03c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:05.493393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96b47-9fe8-4f10-a5e9-6174a63e03c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:05.504666 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb96b47-9fe8-4f10-a5e9-6174a63e03c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:05.506474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb96b47-9fe8-4f10-a5e9-6174a63e03c6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:05.508850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-fcb96b47-9fe8-4f10-a5e9-6174a63e03c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:05.528032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:05.529020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:05.529941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:05.529941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:05.529941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:05.533518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 377/752] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:05.538165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb96b47-9fe8-4f10-a5e9-6174a63e03c6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:05.637282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d2348-1045-499a-b41a-8f19d59459cb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:05.674894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab6874-8ab6-42e3-bf99-384b838cfc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:05.675970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab6874-8ab6-42e3-bf99-384b838cfc18 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:05.676447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab6874-8ab6-42e3-bf99-384b838cfc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:05.676911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab6874-8ab6-42e3-bf99-384b838cfc18 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:05.677780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab6874-8ab6-42e3-bf99-384b838cfc18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:05.678380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab6874-8ab6-42e3-bf99-384b838cfc18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:05.682004 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab6874-8ab6-42e3-bf99-384b838cfc18 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:05.697926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:05.708593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 376/753] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:25:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:05.710317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab6874-8ab6-42e3-bf99-384b838cfc18 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:05.747247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb96b47-9fe8-4f10-a5e9-6174a63e03c6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:05.831941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:05.838453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab6874-8ab6-42e3-bf99-384b838cfc18 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:05.842844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aa452-9ca6-47d8-98d0-7a9924a44f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:05.843476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aa452-9ca6-47d8-98d0-7a9924a44f4a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:05.843763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aa452-9ca6-47d8-98d0-7a9924a44f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:05.847577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aa452-9ca6-47d8-98d0-7a9924a44f4a None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:05.847577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72aa452-9ca6-47d8-98d0-7a9924a44f4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:05.847577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72aa452-9ca6-47d8-98d0-7a9924a44f4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:05.847577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72aa452-9ca6-47d8-98d0-7a9924a44f4a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:05.870328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 378/754] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:05.874410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d456bb-faf8-4dd9-bf0e-a112f6f4d7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:05.875122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d456bb-faf8-4dd9-bf0e-a112f6f4d7bb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:05.875246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aa452-9ca6-47d8-98d0-7a9924a44f4a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:05.875971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d456bb-faf8-4dd9-bf0e-a112f6f4d7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:05.876256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d456bb-faf8-4dd9-bf0e-a112f6f4d7bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:05.887182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d456bb-faf8-4dd9-bf0e-a112f6f4d7bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:05.888969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d456bb-faf8-4dd9-bf0e-a112f6f4d7bb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-87d456bb-faf8-4dd9-bf0e-a112f6f4d7bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:05.905181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:05.906212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:05.907165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:05.907165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:05.907165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:05.907165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 377/755] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:05.912746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d456bb-faf8-4dd9-bf0e-a112f6f4d7bb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:06.066783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d456bb-faf8-4dd9-bf0e-a112f6f4d7bb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:06.105753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb09c9-91b5-4f3e-b0ea-4f4ec105fb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:06.106424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb09c9-91b5-4f3e-b0ea-4f4ec105fb58 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:06.107910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb09c9-91b5-4f3e-b0ea-4f4ec105fb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:06.108189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb09c9-91b5-4f3e-b0ea-4f4ec105fb58 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:06.108832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb09c9-91b5-4f3e-b0ea-4f4ec105fb58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:06.109134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb09c9-91b5-4f3e-b0ea-4f4ec105fb58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:06.111056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb09c9-91b5-4f3e-b0ea-4f4ec105fb58 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:06.116193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aa452-9ca6-47d8-98d0-7a9924a44f4a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:06.138911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 378/756] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:06.147365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb09c9-91b5-4f3e-b0ea-4f4ec105fb58 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:06.164017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:06.186464 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000f15-cc79-4980-8bbf-d00110ae446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:06.187147 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000f15-cc79-4980-8bbf-d00110ae446e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:06.187476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000f15-cc79-4980-8bbf-d00110ae446e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:06.187752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62000f15-cc79-4980-8bbf-d00110ae446e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:06.193538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:06.200532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62000f15-cc79-4980-8bbf-d00110ae446e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:06.201961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62000f15-cc79-4980-8bbf-d00110ae446e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-62000f15-cc79-4980-8bbf-d00110ae446e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:06.213245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:06.214159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:06.215095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:06.215095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:06.215095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:06.215095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:06.215095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:06.215095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:06.215095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:06.215095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:06.215095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 379/757] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:06.220311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000f15-cc79-4980-8bbf-d00110ae446e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:06.330455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb09c9-91b5-4f3e-b0ea-4f4ec105fb58 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:06.380325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461a6c2-463d-4d76-876b-dbc8c835987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:06.380975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461a6c2-463d-4d76-876b-dbc8c835987c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:06.381256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461a6c2-463d-4d76-876b-dbc8c835987c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:06.381531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461a6c2-463d-4d76-876b-dbc8c835987c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:06.393956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461a6c2-463d-4d76-876b-dbc8c835987c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:06.395432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461a6c2-463d-4d76-876b-dbc8c835987c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:06.412830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62000f15-cc79-4980-8bbf-d00110ae446e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1461a6c2-463d-4d76-876b-dbc8c835987c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:06.425745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:06.426750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:06.427712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 379/758] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 05:25:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:06.432302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461a6c2-463d-4d76-876b-dbc8c835987c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:06.477827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49eda4-a98e-4752-ae58-bbe30d5bf1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:06.478471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49eda4-a98e-4752-ae58-bbe30d5bf1ff None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:06.478748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49eda4-a98e-4752-ae58-bbe30d5bf1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:06.479019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49eda4-a98e-4752-ae58-bbe30d5bf1ff None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:06.479723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49eda4-a98e-4752-ae58-bbe30d5bf1ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:06.480027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49eda4-a98e-4752-ae58-bbe30d5bf1ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:06.482179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49eda4-a98e-4752-ae58-bbe30d5bf1ff None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:06.502130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 380/759] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:06.508053 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49eda4-a98e-4752-ae58-bbe30d5bf1ff None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:06.523987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:06.590175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461a6c2-463d-4d76-876b-dbc8c835987c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:06.615583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:06.658346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c6dd8-1945-4d3d-9e14-85b974f6a8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:06.658956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c6dd8-1945-4d3d-9e14-85b974f6a8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:06.659234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c6dd8-1945-4d3d-9e14-85b974f6a8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:06.659500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c6dd8-1945-4d3d-9e14-85b974f6a8d4 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:06.660233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c6dd8-1945-4d3d-9e14-85b974f6a8d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:06.660535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c6dd8-1945-4d3d-9e14-85b974f6a8d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:06.661953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c6dd8-1945-4d3d-9e14-85b974f6a8d4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:06.694354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 380/760] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 05:25:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:06.699171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49eda4-a98e-4752-ae58-bbe30d5bf1ff None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:06.701349 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c6dd8-1945-4d3d-9e14-85b974f6a8d4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:06.762923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6e87d-dca3-4613-ba39-32b988719fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:06.763912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6e87d-dca3-4613-ba39-32b988719fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:06.769535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6e87d-dca3-4613-ba39-32b988719fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:06.770032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6e87d-dca3-4613-ba39-32b988719fc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:06.771017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6e87d-dca3-4613-ba39-32b988719fc7 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=783ad0cd9721409e8a7114cecef16aab) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:06.771530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6e87d-dca3-4613-ba39-32b988719fc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:06.773160 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6e87d-dca3-4613-ba39-32b988719fc7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:06.782607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 381/761] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:06 2022] GET /identity/v3/role_assignments?user.id=783ad0cd9721409e8a7114cecef16aab&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:06.864443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c6dd8-1945-4d3d-9e14-85b974f6a8d4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:06.914205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:06.931943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b0850-f7c3-40e9-8be9-a0cb95e57a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:06.932607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0850-f7c3-40e9-8be9-a0cb95e57a6a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:06.932889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0850-f7c3-40e9-8be9-a0cb95e57a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:06.933164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0850-f7c3-40e9-8be9-a0cb95e57a6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:06.951459 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b0850-f7c3-40e9-8be9-a0cb95e57a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:06.952855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b0850-f7c3-40e9-8be9-a0cb95e57a6a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:06.957060 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4a272-a4db-47cb-98b7-32406f49762e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:06.957663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4a272-a4db-47cb-98b7-32406f49762e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:06.958198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4a272-a4db-47cb-98b7-32406f49762e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:06.961554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 382/762] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:06.966954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459268d-1c39-403a-8dc1-03a2b4db91dd None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:06.967531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459268d-1c39-403a-8dc1-03a2b4db91dd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:06.968007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459268d-1c39-403a-8dc1-03a2b4db91dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2d2b0850-f7c3-40e9-8be9-a0cb95e57a6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:06.980455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:06.981517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:06.983044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:06.983044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:06.983044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:06.983044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 381/763] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:06.983273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5171ea-7507-4a49-8886-40d69230b8c1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:06.983510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5171ea-7507-4a49-8886-40d69230b8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:06.983785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5171ea-7507-4a49-8886-40d69230b8c1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:06.985362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 382/764] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:06.990289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5171ea-7507-4a49-8886-40d69230b8c1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:06.993932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c459268d-1c39-403a-8dc1-03a2b4db91dd None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:06.995309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c459268d-1c39-403a-8dc1-03a2b4db91dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:07.079255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 383/765] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:07.080926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459d1af-d40e-4663-b68a-3b4dc313bf9d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:07.081398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459d1af-d40e-4663-b68a-3b4dc313bf9d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:07.081835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459d1af-d40e-4663-b68a-3b4dc313bf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:07.100525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d459d1af-d40e-4663-b68a-3b4dc313bf9d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:07.102051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d459d1af-d40e-4663-b68a-3b4dc313bf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:07.150515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:07.152915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5171ea-7507-4a49-8886-40d69230b8c1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:07.190125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 384/766] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:07.204479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373530a-edb4-4092-ba27-e313071412ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:07.205365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373530a-edb4-4092-ba27-e313071412ad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:07.205841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373530a-edb4-4092-ba27-e313071412ad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:07.206390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373530a-edb4-4092-ba27-e313071412ad None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:07.207240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373530a-edb4-4092-ba27-e313071412ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:07.207720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373530a-edb4-4092-ba27-e313071412ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:07.209350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373530a-edb4-4092-ba27-e313071412ad None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:07.226780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 383/767] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:07.232039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459d1af-d40e-4663-b68a-3b4dc313bf9d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:07.340059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459d1af-d40e-4663-b68a-3b4dc313bf9d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:07.374849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d517f31-5e51-4119-be39-4e3e6331d535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:07.376451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d517f31-5e51-4119-be39-4e3e6331d535 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:07.376931 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d517f31-5e51-4119-be39-4e3e6331d535 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:07.377382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d517f31-5e51-4119-be39-4e3e6331d535 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:07.384720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d517f31-5e51-4119-be39-4e3e6331d535 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:07.386371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d517f31-5e51-4119-be39-4e3e6331d535 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:07.396048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373530a-edb4-4092-ba27-e313071412ad None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5d517f31-5e51-4119-be39-4e3e6331d535 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:07.398425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:07.399359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:07.400255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:07.400255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:07.400255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:07.400255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:07.400255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:07.400255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:07.400255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:07.400255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:07.402322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 385/768] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:07.411087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d517f31-5e51-4119-be39-4e3e6331d535 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:07.442367 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:07.474446 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:07.547764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d517f31-5e51-4119-be39-4e3e6331d535 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:07.575064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373530a-edb4-4092-ba27-e313071412ad None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:07.592205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666da1c9-6d00-48b7-a6f0-526724a2254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:07.592951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666da1c9-6d00-48b7-a6f0-526724a2254a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:07.593227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666da1c9-6d00-48b7-a6f0-526724a2254a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:07.593738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666da1c9-6d00-48b7-a6f0-526724a2254a None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:07.594878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666da1c9-6d00-48b7-a6f0-526724a2254a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:07.595188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666da1c9-6d00-48b7-a6f0-526724a2254a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:07.596666 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666da1c9-6d00-48b7-a6f0-526724a2254a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:07.612019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec591cf-3de8-4c88-a779-975fe9adcdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:07.612646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec591cf-3de8-4c88-a779-975fe9adcdc2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:07.612920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec591cf-3de8-4c88-a779-975fe9adcdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:07.613187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec591cf-3de8-4c88-a779-975fe9adcdc2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:07.613761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec591cf-3de8-4c88-a779-975fe9adcdc2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:07.615207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec591cf-3de8-4c88-a779-975fe9adcdc2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:07.617888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 386/769] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:07.619518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df20d1a-841c-4a4c-8eb2-d2714c7d0db1 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:07.619815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df20d1a-841c-4a4c-8eb2-d2714c7d0db1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:07.620085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df20d1a-841c-4a4c-8eb2-d2714c7d0db1 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:07.621756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 387/770] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:07.622333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df20d1a-841c-4a4c-8eb2-d2714c7d0db1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3ec591cf-3de8-4c88-a779-975fe9adcdc2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:07.632099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:07.633090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:07.633936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 384/771] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:07 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:07.636210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec591cf-3de8-4c88-a779-975fe9adcdc2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:07.735211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:07.739311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df20d1a-841c-4a4c-8eb2-d2714c7d0db1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:07.740309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec591cf-3de8-4c88-a779-975fe9adcdc2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:07.742245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:07.773199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f5194-2d85-4e94-a931-d655c97bee9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:07.773836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f5194-2d85-4e94-a931-d655c97bee9f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:07.774108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f5194-2d85-4e94-a931-d655c97bee9f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:07.774402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f5194-2d85-4e94-a931-d655c97bee9f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:07.774976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f5194-2d85-4e94-a931-d655c97bee9f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:07.776430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f5194-2d85-4e94-a931-d655c97bee9f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:07.779270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fbf61-e439-4620-b82b-22a3c41588f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:07.780643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fbf61-e439-4620-b82b-22a3c41588f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:07.780643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fbf61-e439-4620-b82b-22a3c41588f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:07.780643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fbf61-e439-4620-b82b-22a3c41588f8 None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/fba4f0473f4c4d3bb2d50b09f3de9837/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:07.783325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58fbf61-e439-4620-b82b-22a3c41588f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=fba4f0473f4c4d3bb2d50b09f3de9837, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:07.784858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58fbf61-e439-4620-b82b-22a3c41588f8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e74f5194-2d85-4e94-a931-d655c97bee9f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:07.787377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:07.788334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:07.789174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 388/772] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Oct 4 05:25:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:07.789967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46a20d-3fe7-439a-bf61-fb7225b94036 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:07.789967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46a20d-3fe7-439a-bf61-fb7225b94036 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:07.790089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46a20d-3fe7-439a-bf61-fb7225b94036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:07.808447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bc46a20d-3fe7-439a-bf61-fb7225b94036 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:07.809789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bc46a20d-3fe7-439a-bf61-fb7225b94036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:07.856872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 385/773] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:25:07 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/fba4f0473f4c4d3bb2d50b09f3de9837/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:25:07.867257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fbf61-e439-4620-b82b-22a3c41588f8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:07.902627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 389/774] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:07.903670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46a20d-3fe7-439a-bf61-fb7225b94036 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:08.016722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fbf61-e439-4620-b82b-22a3c41588f8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:08.059857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a1964-635b-4c8c-b100-22d1a7f73d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:08.061452 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a1964-635b-4c8c-b100-22d1a7f73d15 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:08.062279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a1964-635b-4c8c-b100-22d1a7f73d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:08.063295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a1964-635b-4c8c-b100-22d1a7f73d15 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:08.064149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a1964-635b-4c8c-b100-22d1a7f73d15 None admin] RBAC: Adding query filter params (name=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:08.064718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a1964-635b-4c8c-b100-22d1a7f73d15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:08.066638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a1964-635b-4c8c-b100-22d1a7f73d15 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:08.078295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 386/775] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:08.082412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a1964-635b-4c8c-b100-22d1a7f73d15 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:08.105968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:08.114125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46a20d-3fe7-439a-bf61-fb7225b94036 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:08.121600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:08.150552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bf488-197e-4db1-ac3d-686c1edcd79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:08.151298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bf488-197e-4db1-ac3d-686c1edcd79c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:08.151640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bf488-197e-4db1-ac3d-686c1edcd79c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:08.152067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bf488-197e-4db1-ac3d-686c1edcd79c None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:08.152705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bf488-197e-4db1-ac3d-686c1edcd79c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:08.153075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bf488-197e-4db1-ac3d-686c1edcd79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:08.154595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bf488-197e-4db1-ac3d-686c1edcd79c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:08.164166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 390/776] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 05:25:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:08.166044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bf488-197e-4db1-ac3d-686c1edcd79c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:08.216541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a1964-635b-4c8c-b100-22d1a7f73d15 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:08.253117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de1ffc-8e38-4c6c-8c8f-b41abc520005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:08.253784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de1ffc-8e38-4c6c-8c8f-b41abc520005 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:08.254054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de1ffc-8e38-4c6c-8c8f-b41abc520005 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:08.254368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de1ffc-8e38-4c6c-8c8f-b41abc520005 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:08.254991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de1ffc-8e38-4c6c-8c8f-b41abc520005 None admin] RBAC: Adding query filter params (type=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:08.255296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de1ffc-8e38-4c6c-8c8f-b41abc520005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:08.256677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de1ffc-8e38-4c6c-8c8f-b41abc520005 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:08.262177 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 387/777] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:08.268092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de1ffc-8e38-4c6c-8c8f-b41abc520005 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:08.303580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bf488-197e-4db1-ac3d-686c1edcd79c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:08.339578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e68c0-3506-40b7-83fb-ce8a856b346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:08.340586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e68c0-3506-40b7-83fb-ce8a856b346b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:08.341083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e68c0-3506-40b7-83fb-ce8a856b346b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:08.341565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e68c0-3506-40b7-83fb-ce8a856b346b None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:08.342456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e68c0-3506-40b7-83fb-ce8a856b346b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:08.342957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e68c0-3506-40b7-83fb-ce8a856b346b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:08.344563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e68c0-3506-40b7-83fb-ce8a856b346b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:08.353506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 391/778] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 05:25:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:08.354639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e68c0-3506-40b7-83fb-ce8a856b346b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:08.385634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:08.409840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:08.421920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de1ffc-8e38-4c6c-8c8f-b41abc520005 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:08.466768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e68c0-3506-40b7-83fb-ce8a856b346b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:08.467013 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ade450-3374-4475-9c06-d4c256762893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:08.467893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ade450-3374-4475-9c06-d4c256762893 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:08.468323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ade450-3374-4475-9c06-d4c256762893 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:08.468912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ade450-3374-4475-9c06-d4c256762893 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:08.470457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ade450-3374-4475-9c06-d4c256762893 None admin] RBAC: Adding query filter params (interface=public, service_id=60e7a14a6d6d45c392874ddfea2e496c, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:08.470777 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ade450-3374-4475-9c06-d4c256762893 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:08.472134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ade450-3374-4475-9c06-d4c256762893 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:08.485131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 388/779] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 05:25:08 2022] GET /identity/v3/endpoints?service_id=60e7a14a6d6d45c392874ddfea2e496c&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:08.485131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ade450-3374-4475-9c06-d4c256762893 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:08.514804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324a0eb-01c5-4204-a3fd-61dca818fba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:08.515656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324a0eb-01c5-4204-a3fd-61dca818fba8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:08.516058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324a0eb-01c5-4204-a3fd-61dca818fba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:08.516517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324a0eb-01c5-4204-a3fd-61dca818fba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:08.527672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324a0eb-01c5-4204-a3fd-61dca818fba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d324a0eb-01c5-4204-a3fd-61dca818fba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:08.533285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:08.534507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:25:08.535568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:08.537383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 392/780] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:25:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:08.538282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324a0eb-01c5-4204-a3fd-61dca818fba8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:08.616307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ade450-3374-4475-9c06-d4c256762893 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:08.624739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:08.644070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324a0eb-01c5-4204-a3fd-61dca818fba8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:08.661701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9a788-4809-42a7-896d-aa43468cb7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:08.662765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9a788-4809-42a7-896d-aa43468cb7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:08.663197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9a788-4809-42a7-896d-aa43468cb7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:08.663910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9a788-4809-42a7-896d-aa43468cb7b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:08.664782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9a788-4809-42a7-896d-aa43468cb7b4 None admin] RBAC: Adding query filter params (interface=public, service_id=e91a606028a743b984e62b19cf51c0c3, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:08.665287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9a788-4809-42a7-896d-aa43468cb7b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:08.667435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9a788-4809-42a7-896d-aa43468cb7b4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:08.676045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 389/781] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 05:25:08 2022] GET /identity/v3/endpoints?service_id=e91a606028a743b984e62b19cf51c0c3&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:08.677337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9a788-4809-42a7-896d-aa43468cb7b4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:08.688984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadf29e-5e08-4127-a0a1-4cd24e12d7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:08.690040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadf29e-5e08-4127-a0a1-4cd24e12d7ef None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:08.690652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadf29e-5e08-4127-a0a1-4cd24e12d7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:08.691488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadf29e-5e08-4127-a0a1-4cd24e12d7ef None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:08.692634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadf29e-5e08-4127-a0a1-4cd24e12d7ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:08.695335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadf29e-5e08-4127-a0a1-4cd24e12d7ef None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:08.702100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:08.704731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 393/782] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 05:25:08 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:08.705731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadf29e-5e08-4127-a0a1-4cd24e12d7ef None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:08.854254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadf29e-5e08-4127-a0a1-4cd24e12d7ef None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:08.879881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9a788-4809-42a7-896d-aa43468cb7b4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:08.907280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad51eb-cca0-473e-8d05-bbf80671fca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:08.908340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad51eb-cca0-473e-8d05-bbf80671fca4 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:08.908771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad51eb-cca0-473e-8d05-bbf80671fca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:08.909193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad51eb-cca0-473e-8d05-bbf80671fca4 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:08.910226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad51eb-cca0-473e-8d05-bbf80671fca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:08.912016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad51eb-cca0-473e-8d05-bbf80671fca4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:08.920435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 394/783] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 05:25:08 2022] GET /identity/v3/services => generated 772 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:08.962762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee7a42-f8da-4c4f-95ce-b43859f7b821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:08.963396 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee7a42-f8da-4c4f-95ce-b43859f7b821 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:08.963670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee7a42-f8da-4c4f-95ce-b43859f7b821 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:08.963940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee7a42-f8da-4c4f-95ce-b43859f7b821 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:08.964721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee7a42-f8da-4c4f-95ce-b43859f7b821 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:08.965027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee7a42-f8da-4c4f-95ce-b43859f7b821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:08.966516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee7a42-f8da-4c4f-95ce-b43859f7b821 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:08.979821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 390/784] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:25:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:08.985650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad51eb-cca0-473e-8d05-bbf80671fca4 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:09.004472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:09.114659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad51eb-cca0-473e-8d05-bbf80671fca4 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:09.151304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76cbe7-9e72-415b-a40d-beaa58177563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:09.151976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76cbe7-9e72-415b-a40d-beaa58177563 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:09.152257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76cbe7-9e72-415b-a40d-beaa58177563 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:09.152534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76cbe7-9e72-415b-a40d-beaa58177563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:09.165932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76cbe7-9e72-415b-a40d-beaa58177563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:09.167479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76cbe7-9e72-415b-a40d-beaa58177563 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9d76cbe7-9e72-415b-a40d-beaa58177563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:09.185696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:09.186746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:09.187633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:09.187633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:09.187633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:09.187633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 395/785] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:09.192797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee7a42-f8da-4c4f-95ce-b43859f7b821 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:09.282072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:09.288399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148ba16-d419-4574-adfa-199cc827fdd5 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:09.288685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148ba16-d419-4574-adfa-199cc827fdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:09.288961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148ba16-d419-4574-adfa-199cc827fdd5 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:09.290327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 396/786] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:09.295191 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed198215-a17a-4733-927b-72c4e0bd9b29 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:09.295480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed198215-a17a-4733-927b-72c4e0bd9b29 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:09.295756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed198215-a17a-4733-927b-72c4e0bd9b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:09.315172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ed198215-a17a-4733-927b-72c4e0bd9b29 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:09.316368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ed198215-a17a-4733-927b-72c4e0bd9b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:09.321979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee7a42-f8da-4c4f-95ce-b43859f7b821 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:09.386975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b4465-ac53-4de9-91f8-23cd32ab0fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:09.387506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b4465-ac53-4de9-91f8-23cd32ab0fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:09.387784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b4465-ac53-4de9-91f8-23cd32ab0fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:09.388054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b4465-ac53-4de9-91f8-23cd32ab0fe6 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:09.388697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b4465-ac53-4de9-91f8-23cd32ab0fe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:09.388999 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b4465-ac53-4de9-91f8-23cd32ab0fe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:09.390388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b4465-ac53-4de9-91f8-23cd32ab0fe6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:09.410267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 391/787] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:09.415061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:09.416962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b4465-ac53-4de9-91f8-23cd32ab0fe6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:09.435145 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 397/788] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:09.532522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b4465-ac53-4de9-91f8-23cd32ab0fe6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:09.567217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad9f81-89f1-471e-8ed2-8d173c2dd6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:09.567856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad9f81-89f1-471e-8ed2-8d173c2dd6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:09.568136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad9f81-89f1-471e-8ed2-8d173c2dd6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:09.568410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad9f81-89f1-471e-8ed2-8d173c2dd6d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:09.576630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad9f81-89f1-471e-8ed2-8d173c2dd6d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:09.578074 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad9f81-89f1-471e-8ed2-8d173c2dd6d8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:09.581510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-27ad9f81-89f1-471e-8ed2-8d173c2dd6d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:09.590740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:09.600951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:09.601904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 392/789] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 05:25:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:09.602520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed198215-a17a-4733-927b-72c4e0bd9b29 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:09.743840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed198215-a17a-4733-927b-72c4e0bd9b29 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:09.791704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46c1b6-ad88-479f-b10e-6ecbf465c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:09.792534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46c1b6-ad88-479f-b10e-6ecbf465c5cd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:09.793057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46c1b6-ad88-479f-b10e-6ecbf465c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:09.793466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46c1b6-ad88-479f-b10e-6ecbf465c5cd None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:09.794354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46c1b6-ad88-479f-b10e-6ecbf465c5cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:09.794814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46c1b6-ad88-479f-b10e-6ecbf465c5cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:09.796360 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46c1b6-ad88-479f-b10e-6ecbf465c5cd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:09.816424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad9f81-89f1-471e-8ed2-8d173c2dd6d8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:09.828257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 398/790] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 05:25:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:09.834197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46c1b6-ad88-479f-b10e-6ecbf465c5cd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:09.969023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:10.032222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46c1b6-ad88-479f-b10e-6ecbf465c5cd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:10.079211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dcecb-0215-4079-9492-598b409f7db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:10.081317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dcecb-0215-4079-9492-598b409f7db5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:10.081853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dcecb-0215-4079-9492-598b409f7db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:10.082443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dcecb-0215-4079-9492-598b409f7db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:10.096780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad9f81-89f1-471e-8ed2-8d173c2dd6d8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:10.099319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024dcecb-0215-4079-9492-598b409f7db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:10.101693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024dcecb-0215-4079-9492-598b409f7db5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-024dcecb-0215-4079-9492-598b409f7db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:10.120130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:10.121298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:10.122448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:10.122448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:10.122448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:10.124891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 399/791] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:10.130886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dcecb-0215-4079-9492-598b409f7db5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:10.190884 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f25db7-7182-4dc2-aebd-040561def64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:10.191572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f25db7-7182-4dc2-aebd-040561def64c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:10.191858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f25db7-7182-4dc2-aebd-040561def64c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:10.192139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f25db7-7182-4dc2-aebd-040561def64c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:10.215817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f25db7-7182-4dc2-aebd-040561def64c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-63f25db7-7182-4dc2-aebd-040561def64c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:10.230499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:10.231497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:10.232321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 393/792] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:10.238550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f25db7-7182-4dc2-aebd-040561def64c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:10.249418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dcecb-0215-4079-9492-598b409f7db5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:10.251703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:10.289756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4525129-61da-4224-89f9-dfe7523ecafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:10.290793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4525129-61da-4224-89f9-dfe7523ecafc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:10.291335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4525129-61da-4224-89f9-dfe7523ecafc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:10.291817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4525129-61da-4224-89f9-dfe7523ecafc None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:10.292661 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4525129-61da-4224-89f9-dfe7523ecafc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:10.293158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4525129-61da-4224-89f9-dfe7523ecafc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:10.294818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4525129-61da-4224-89f9-dfe7523ecafc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:10.308623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 400/793] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:10.320148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4525129-61da-4224-89f9-dfe7523ecafc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:10.376148 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:10.449755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4525129-61da-4224-89f9-dfe7523ecafc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:10.472619 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f25db7-7182-4dc2-aebd-040561def64c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:10.513856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6b256-d257-46cd-9f19-6a74eb5adfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:10.518618 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6b256-d257-46cd-9f19-6a74eb5adfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:10.519366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6b256-d257-46cd-9f19-6a74eb5adfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:10.520756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6b256-d257-46cd-9f19-6a74eb5adfb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:10.532419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6b256-d257-46cd-9f19-6a74eb5adfb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:10.534314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6b256-d257-46cd-9f19-6a74eb5adfb0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-38e6b256-d257-46cd-9f19-6a74eb5adfb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:10.552032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:10.552941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:10.553867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:10.553867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:10.553867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:10.553867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:10.553867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:10.553867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:10.553867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:10.553867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:10.560453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba180614-c266-47f3-9928-848a4e9e7d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:10.561378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba180614-c266-47f3-9928-848a4e9e7d89 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:10.561852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba180614-c266-47f3-9928-848a4e9e7d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:10.562342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba180614-c266-47f3-9928-848a4e9e7d89 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:10.563296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba180614-c266-47f3-9928-848a4e9e7d89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:10.563794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba180614-c266-47f3-9928-848a4e9e7d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:10.565339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba180614-c266-47f3-9928-848a4e9e7d89 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:10.567193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 401/794] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:10.568322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6b256-d257-46cd-9f19-6a74eb5adfb0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:10.592429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 394/795] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:25:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:10.598326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba180614-c266-47f3-9928-848a4e9e7d89 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:10.617036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:10.726292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6b256-d257-46cd-9f19-6a74eb5adfb0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:10.764916 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fade44-0831-4f76-b5ca-3f0b3580cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:10.766542 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fade44-0831-4f76-b5ca-3f0b3580cc64 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:10.767045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fade44-0831-4f76-b5ca-3f0b3580cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:10.767509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fade44-0831-4f76-b5ca-3f0b3580cc64 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:10.768489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fade44-0831-4f76-b5ca-3f0b3580cc64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:10.769077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fade44-0831-4f76-b5ca-3f0b3580cc64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:10.770723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fade44-0831-4f76-b5ca-3f0b3580cc64 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:10.771784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba180614-c266-47f3-9928-848a4e9e7d89 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:10.784342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 402/796] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:10.786218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fa87a-0e95-47b2-9ae1-77d95e837b7f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:10.786734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fa87a-0e95-47b2-9ae1-77d95e837b7f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:10.787213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fa87a-0e95-47b2-9ae1-77d95e837b7f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:10.789248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 403/797] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:10.790202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fa87a-0e95-47b2-9ae1-77d95e837b7f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:10.799141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:10.817476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb8f1e-fead-4707-a413-65880f3ae5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:10.817476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb8f1e-fead-4707-a413-65880f3ae5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:10.817476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb8f1e-fead-4707-a413-65880f3ae5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:10.817476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb8f1e-fead-4707-a413-65880f3ae5f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:10.823758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb8f1e-fead-4707-a413-65880f3ae5f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:10.825157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb8f1e-fead-4707-a413-65880f3ae5f2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ecfb8f1e-fead-4707-a413-65880f3ae5f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:10.840346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:10.841357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:10.842277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:10.842277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:10.842277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:10.845718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 395/798] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:10.850890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831223d6-e770-4856-8e4b-33067727edf5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:10.850890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831223d6-e770-4856-8e4b-33067727edf5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:10.850890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831223d6-e770-4856-8e4b-33067727edf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:10.871125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-831223d6-e770-4856-8e4b-33067727edf5 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:10.872259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-831223d6-e770-4856-8e4b-33067727edf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:10.903512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:10.926131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fa87a-0e95-47b2-9ae1-77d95e837b7f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:10.946255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 396/799] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:10.947148 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831223d6-e770-4856-8e4b-33067727edf5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:10.965973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b0cfe-a3aa-4575-9d3d-3151a431b9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:10.966997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b0cfe-a3aa-4575-9d3d-3151a431b9b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:10.967540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b0cfe-a3aa-4575-9d3d-3151a431b9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:10.968079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b0cfe-a3aa-4575-9d3d-3151a431b9b8 None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/5cf157961e3d45d5ad861f9b4f51ab50/roles/bc27c203c0234a6194c03f93ee3c50a9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:10.984605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b0cfe-a3aa-4575-9d3d-3151a431b9b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=5cf157961e3d45d5ad861f9b4f51ab50, role_id=bc27c203c0234a6194c03f93ee3c50a9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:10.986521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b0cfe-a3aa-4575-9d3d-3151a431b9b8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:11.072421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 404/800] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:25:10 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/5cf157961e3d45d5ad861f9b4f51ab50/roles/bc27c203c0234a6194c03f93ee3c50a9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:25:11.090453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831223d6-e770-4856-8e4b-33067727edf5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:11.093674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:11.131670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595bef1e-3da7-4efd-9910-afdd3ff06f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:11.132782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bef1e-3da7-4efd-9910-afdd3ff06f78 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:11.133259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bef1e-3da7-4efd-9910-afdd3ff06f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:11.133805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bef1e-3da7-4efd-9910-afdd3ff06f78 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:11.134866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595bef1e-3da7-4efd-9910-afdd3ff06f78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:11.135394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595bef1e-3da7-4efd-9910-afdd3ff06f78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:11.137133 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595bef1e-3da7-4efd-9910-afdd3ff06f78 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:11.152205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 397/801] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:11.158741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b0cfe-a3aa-4575-9d3d-3151a431b9b8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:11.242365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595bef1e-3da7-4efd-9910-afdd3ff06f78 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:11.290117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b0cfe-a3aa-4575-9d3d-3151a431b9b8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:11.324695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:11.330174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63a9fe-94fa-493f-81d0-ab7b5af83cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:11.330758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63a9fe-94fa-493f-81d0-ab7b5af83cbf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:11.331037 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63a9fe-94fa-493f-81d0-ab7b5af83cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:11.331310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63a9fe-94fa-493f-81d0-ab7b5af83cbf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:11.340906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63a9fe-94fa-493f-81d0-ab7b5af83cbf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:11.341653 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63a9fe-94fa-493f-81d0-ab7b5af83cbf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8b63a9fe-94fa-493f-81d0-ab7b5af83cbf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:11.354245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:11.355256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:11.356239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 405/802] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:11.356832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb155c36-6b81-4f1d-a7ce-a24953e65a70 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:11.356907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb155c36-6b81-4f1d-a7ce-a24953e65a70 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:11.357491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb155c36-6b81-4f1d-a7ce-a24953e65a70 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:11.362169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 406/803] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:11.362713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb155c36-6b81-4f1d-a7ce-a24953e65a70 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:11.383492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595bef1e-3da7-4efd-9910-afdd3ff06f78 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:11.418861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfa1fe-546c-4056-92b6-a2882c8b86d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:11.419592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfa1fe-546c-4056-92b6-a2882c8b86d4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:11.419960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfa1fe-546c-4056-92b6-a2882c8b86d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:11.420299 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfa1fe-546c-4056-92b6-a2882c8b86d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:11.432909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfa1fe-546c-4056-92b6-a2882c8b86d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-53dfa1fe-546c-4056-92b6-a2882c8b86d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:11.439941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:11.440924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:11.441789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 398/804] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:11.442550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a414232-e004-48d9-907e-65cfe8fc0045 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:11.443307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a414232-e004-48d9-907e-65cfe8fc0045 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:11.443677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a414232-e004-48d9-907e-65cfe8fc0045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:11.461982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8a414232-e004-48d9-907e-65cfe8fc0045 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:11.463395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8a414232-e004-48d9-907e-65cfe8fc0045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:11.484059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:11.495145 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb155c36-6b81-4f1d-a7ce-a24953e65a70 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:11.542783 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83792368-8622-4a33-9664-e74d68af9b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:11.543556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83792368-8622-4a33-9664-e74d68af9b63 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:11.543941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83792368-8622-4a33-9664-e74d68af9b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:11.544087 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 399/805] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:11.550161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83792368-8622-4a33-9664-e74d68af9b63 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:11.550161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83792368-8622-4a33-9664-e74d68af9b63 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:11.550161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83792368-8622-4a33-9664-e74d68af9b63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:11.550161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83792368-8622-4a33-9664-e74d68af9b63 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:11.551395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a414232-e004-48d9-907e-65cfe8fc0045 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:11.568215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 407/806] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 05:25:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:11.574311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83792368-8622-4a33-9664-e74d68af9b63 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:11.678560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:11.689606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83792368-8622-4a33-9664-e74d68af9b63 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:11.742014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fc08b-2756-416d-8034-8bbbb414e977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:11.742692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fc08b-2756-416d-8034-8bbbb414e977 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:11.742976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fc08b-2756-416d-8034-8bbbb414e977 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:11.743252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fc08b-2756-416d-8034-8bbbb414e977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:11.766778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fc08b-2756-416d-8034-8bbbb414e977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:11.768235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fc08b-2756-416d-8034-8bbbb414e977 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:11.791013 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a414232-e004-48d9-907e-65cfe8fc0045 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4e5fc08b-2756-416d-8034-8bbbb414e977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:11.800162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:11.801658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:11.802798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:11.802798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:11.802798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:11.802798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 408/807] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:11.808985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261961a1-9542-45ff-be03-55ca1baa3024 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:11.809440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261961a1-9542-45ff-be03-55ca1baa3024 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:11.809885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261961a1-9542-45ff-be03-55ca1baa3024 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:11.812647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 409/808] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:11.816804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6918393-1d21-4a53-80ab-a07fcfebd41d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:11.817264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6918393-1d21-4a53-80ab-a07fcfebd41d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:11.817689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6918393-1d21-4a53-80ab-a07fcfebd41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:11.837865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f6918393-1d21-4a53-80ab-a07fcfebd41d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:11.839316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f6918393-1d21-4a53-80ab-a07fcfebd41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:11.841768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f366c55-79c3-4701-b3d5-9bc91c5abc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:11.843171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f366c55-79c3-4701-b3d5-9bc91c5abc05 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:11.843477 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f366c55-79c3-4701-b3d5-9bc91c5abc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:11.843760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f366c55-79c3-4701-b3d5-9bc91c5abc05 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:11.844957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f366c55-79c3-4701-b3d5-9bc91c5abc05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:11.845271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f366c55-79c3-4701-b3d5-9bc91c5abc05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:11.846729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f366c55-79c3-4701-b3d5-9bc91c5abc05 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:11.857063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 400/809] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:25:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:11.857905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f366c55-79c3-4701-b3d5-9bc91c5abc05 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:11.883334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:11.923773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 410/810] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:11.925077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6918393-1d21-4a53-80ab-a07fcfebd41d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:11.968609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:11.982391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f366c55-79c3-4701-b3d5-9bc91c5abc05 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.024540 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1c99c-9c9a-4835-85d9-bd3d450e683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.025154 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1c99c-9c9a-4835-85d9-bd3d450e683c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.025433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1c99c-9c9a-4835-85d9-bd3d450e683c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.025706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1c99c-9c9a-4835-85d9-bd3d450e683c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.027216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1c99c-9c9a-4835-85d9-bd3d450e683c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:12.027526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1c99c-9c9a-4835-85d9-bd3d450e683c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.028891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1c99c-9c9a-4835-85d9-bd3d450e683c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.046291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 401/811] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:12.047439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1c99c-9c9a-4835-85d9-bd3d450e683c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.066833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6918393-1d21-4a53-80ab-a07fcfebd41d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.114030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64dbbc-717a-4c5f-a3b6-75635ae53e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.115696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64dbbc-717a-4c5f-a3b6-75635ae53e20 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.116513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64dbbc-717a-4c5f-a3b6-75635ae53e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.116942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64dbbc-717a-4c5f-a3b6-75635ae53e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.132246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64dbbc-717a-4c5f-a3b6-75635ae53e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.134287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64dbbc-717a-4c5f-a3b6-75635ae53e20 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.145284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4a64dbbc-717a-4c5f-a3b6-75635ae53e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:12.157523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:12.158697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:12.159817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:12.159817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:12.159817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:12.161953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 411/812] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:12.163563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64dbbc-717a-4c5f-a3b6-75635ae53e20 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.172216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1c99c-9c9a-4835-85d9-bd3d450e683c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.213275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e9fbc-87a5-47e9-87a6-819312be85b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.213909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e9fbc-87a5-47e9-87a6-819312be85b7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.214210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e9fbc-87a5-47e9-87a6-819312be85b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.214479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e9fbc-87a5-47e9-87a6-819312be85b7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.215051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e9fbc-87a5-47e9-87a6-819312be85b7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.216832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e9fbc-87a5-47e9-87a6-819312be85b7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c37e9fbc-87a5-47e9-87a6-819312be85b7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:12.226890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:12.227865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:12.228706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 402/813] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:12 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:12.229498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e9fbc-87a5-47e9-87a6-819312be85b7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.268469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:12.284600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64dbbc-717a-4c5f-a3b6-75635ae53e20 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.321059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1d4fe-1641-4a2c-869f-065ff872d2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.321691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1d4fe-1641-4a2c-869f-065ff872d2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.321973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1d4fe-1641-4a2c-869f-065ff872d2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.322272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1d4fe-1641-4a2c-869f-065ff872d2a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.330252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1d4fe-1641-4a2c-869f-065ff872d2a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.331693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1d4fe-1641-4a2c-869f-065ff872d2a0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.332581 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e9fbc-87a5-47e9-87a6-819312be85b7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-3db1d4fe-1641-4a2c-869f-065ff872d2a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:12.343504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:12.344616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:12.345523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:12.345523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:12.345523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:12.345523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:12.345523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:12.345523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:12.345523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:12.345523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:12.345523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 412/814] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:12.346019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1d4fe-1641-4a2c-869f-065ff872d2a0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.374608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206fab0-4e84-4f65-b9e7-b4733b4bedd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.375760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206fab0-4e84-4f65-b9e7-b4733b4bedd2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.376113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206fab0-4e84-4f65-b9e7-b4733b4bedd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.376286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206fab0-4e84-4f65-b9e7-b4733b4bedd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.376955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206fab0-4e84-4f65-b9e7-b4733b4bedd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:12.377205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206fab0-4e84-4f65-b9e7-b4733b4bedd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.378575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206fab0-4e84-4f65-b9e7-b4733b4bedd2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.388402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:12.396486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 403/815] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:12.396486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206fab0-4e84-4f65-b9e7-b4733b4bedd2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.459569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1d4fe-1641-4a2c-869f-065ff872d2a0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.492501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829507-8358-48b1-9e03-05fe54ab192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.493048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829507-8358-48b1-9e03-05fe54ab192f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.493439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829507-8358-48b1-9e03-05fe54ab192f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.493598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829507-8358-48b1-9e03-05fe54ab192f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.494195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6829507-8358-48b1-9e03-05fe54ab192f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.495614 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6829507-8358-48b1-9e03-05fe54ab192f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.500971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206fab0-4e84-4f65-b9e7-b4733b4bedd2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-d6829507-8358-48b1-9e03-05fe54ab192f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:12.504917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:12.505874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:12.507316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 413/816] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Oct 4 05:25:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:12.508110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829507-8358-48b1-9e03-05fe54ab192f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.518966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:12.540669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524df94-e6fc-45ec-bee9-21f5230ce295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.541275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524df94-e6fc-45ec-bee9-21f5230ce295 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.541551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524df94-e6fc-45ec-bee9-21f5230ce295 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.541818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524df94-e6fc-45ec-bee9-21f5230ce295 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.542469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524df94-e6fc-45ec-bee9-21f5230ce295 None admin] RBAC: Adding query filter params (name=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:12.542774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524df94-e6fc-45ec-bee9-21f5230ce295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.544177 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524df94-e6fc-45ec-bee9-21f5230ce295 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.553162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 404/817] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:12.553162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524df94-e6fc-45ec-bee9-21f5230ce295 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.612450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829507-8358-48b1-9e03-05fe54ab192f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.616188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:12.646513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5714e-f8e1-42a2-803d-2ce5ae0545c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.647119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5714e-f8e1-42a2-803d-2ce5ae0545c1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.647397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5714e-f8e1-42a2-803d-2ce5ae0545c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.647676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5714e-f8e1-42a2-803d-2ce5ae0545c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.648343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5714e-f8e1-42a2-803d-2ce5ae0545c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:12.648645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5714e-f8e1-42a2-803d-2ce5ae0545c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.650061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5714e-f8e1-42a2-803d-2ce5ae0545c1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.655923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524df94-e6fc-45ec-bee9-21f5230ce295 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.666211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 414/818] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:12.666211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5714e-f8e1-42a2-803d-2ce5ae0545c1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.690873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109bf67-a0a3-45b4-a4b9-48efd682ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.691502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109bf67-a0a3-45b4-a4b9-48efd682ecd3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.691791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109bf67-a0a3-45b4-a4b9-48efd682ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.692072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109bf67-a0a3-45b4-a4b9-48efd682ecd3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.699906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109bf67-a0a3-45b4-a4b9-48efd682ecd3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.701359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109bf67-a0a3-45b4-a4b9-48efd682ecd3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9109bf67-a0a3-45b4-a4b9-48efd682ecd3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:12.712370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:12.713298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 405/819] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 05:25:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:12.714285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109bf67-a0a3-45b4-a4b9-48efd682ecd3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.740259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:12.772647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5714e-f8e1-42a2-803d-2ce5ae0545c1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.806639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b7368-4ae7-48d8-9a50-fb11919d74f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.807254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b7368-4ae7-48d8-9a50-fb11919d74f2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.807549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b7368-4ae7-48d8-9a50-fb11919d74f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.807805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b7368-4ae7-48d8-9a50-fb11919d74f2 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.808431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b7368-4ae7-48d8-9a50-fb11919d74f2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:12.808734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b7368-4ae7-48d8-9a50-fb11919d74f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.810090 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b7368-4ae7-48d8-9a50-fb11919d74f2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.814972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 415/820] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 05:25:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:12.815721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b7368-4ae7-48d8-9a50-fb11919d74f2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.831336 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109bf67-a0a3-45b4-a4b9-48efd682ecd3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.860988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:12.872527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40102978-2f52-470a-87c4-f179ab3ea537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.873243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40102978-2f52-470a-87c4-f179ab3ea537 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.873519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40102978-2f52-470a-87c4-f179ab3ea537 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.875100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40102978-2f52-470a-87c4-f179ab3ea537 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.875100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40102978-2f52-470a-87c4-f179ab3ea537 None admin] RBAC: Adding query filter params (type=network) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:12.875100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40102978-2f52-470a-87c4-f179ab3ea537 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.877076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40102978-2f52-470a-87c4-f179ab3ea537 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.886638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 406/821] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:12.886638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40102978-2f52-470a-87c4-f179ab3ea537 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:12.926112 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b7368-4ae7-48d8-9a50-fb11919d74f2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:12.957361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:12.962633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dfb98-860e-430c-bb47-cd9ddc843f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:12.963299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfb98-860e-430c-bb47-cd9ddc843f40 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:12.963573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfb98-860e-430c-bb47-cd9ddc843f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:12.963975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfb98-860e-430c-bb47-cd9ddc843f40 None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/783ad0cd9721409e8a7114cecef16aab/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:12.965870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dfb98-860e-430c-bb47-cd9ddc843f40 None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=783ad0cd9721409e8a7114cecef16aab, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:12.967375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dfb98-860e-430c-bb47-cd9ddc843f40 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:12.994245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40102978-2f52-470a-87c4-f179ab3ea537 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:13.028456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae67d61-e14c-4bef-8f48-ef5f54e6ecb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:13.029017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae67d61-e14c-4bef-8f48-ef5f54e6ecb5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:13.029296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae67d61-e14c-4bef-8f48-ef5f54e6ecb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:13.029570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae67d61-e14c-4bef-8f48-ef5f54e6ecb5 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:13.030385 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae67d61-e14c-4bef-8f48-ef5f54e6ecb5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:13.030691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae67d61-e14c-4bef-8f48-ef5f54e6ecb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:13.032813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae67d61-e14c-4bef-8f48-ef5f54e6ecb5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:13.050937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 407/822] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 05:25:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:13.051755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae67d61-e14c-4bef-8f48-ef5f54e6ecb5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:13.066171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 416/823] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:25:12 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/783ad0cd9721409e8a7114cecef16aab/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:25:13.066171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dfb98-860e-430c-bb47-cd9ddc843f40 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:13.127465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:13.188178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dfb98-860e-430c-bb47-cd9ddc843f40 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:13.200306 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae67d61-e14c-4bef-8f48-ef5f54e6ecb5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:13.238783 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340f773-aaf1-44d0-a8e4-77c599fae086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:13.239501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340f773-aaf1-44d0-a8e4-77c599fae086 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:13.239784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340f773-aaf1-44d0-a8e4-77c599fae086 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:13.240102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340f773-aaf1-44d0-a8e4-77c599fae086 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:13.240827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340f773-aaf1-44d0-a8e4-77c599fae086 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:13.241229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5a767-5d57-4e78-8f16-dc5814a01596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:13.241900 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5a767-5d57-4e78-8f16-dc5814a01596 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:13.242214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340f773-aaf1-44d0-a8e4-77c599fae086 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:13.242329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5a767-5d57-4e78-8f16-dc5814a01596 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:13.242491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5a767-5d57-4e78-8f16-dc5814a01596 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:13.243118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5a767-5d57-4e78-8f16-dc5814a01596 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:13.243421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5a767-5d57-4e78-8f16-dc5814a01596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:13.244762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5a767-5d57-4e78-8f16-dc5814a01596 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:13.250971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 408/824] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 05:25:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:13.251916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5a767-5d57-4e78-8f16-dc5814a01596 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:13.346741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 417/825] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:25:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:13.351270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340f773-aaf1-44d0-a8e4-77c599fae086 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:13.366688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5a767-5d57-4e78-8f16-dc5814a01596 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:13.380812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:13.421658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec019b-187c-4631-9c30-aa1543a37f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:13.422382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec019b-187c-4631-9c30-aa1543a37f07 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:13.422674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec019b-187c-4631-9c30-aa1543a37f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:13.422984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec019b-187c-4631-9c30-aa1543a37f07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:13.438032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:13.510784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340f773-aaf1-44d0-a8e4-77c599fae086 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:13.513977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec019b-187c-4631-9c30-aa1543a37f07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:13.515541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec019b-187c-4631-9c30-aa1543a37f07 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:13.565844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cd599-11de-4862-9e2f-1dc78f3b4f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:13.567112 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cd599-11de-4862-9e2f-1dc78f3b4f40 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:13.567390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cd599-11de-4862-9e2f-1dc78f3b4f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:13.567923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cd599-11de-4862-9e2f-1dc78f3b4f40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:13.568612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cd599-11de-4862-9e2f-1dc78f3b4f40 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:13.570924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cd599-11de-4862-9e2f-1dc78f3b4f40 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:13.625608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 409/826] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:25:13.626831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86df4-6c0c-487f-9919-3fce1f7f53a4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:13.627164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86df4-6c0c-487f-9919-3fce1f7f53a4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:13.627444 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed86df4-6c0c-487f-9919-3fce1f7f53a4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:13.628691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 410/827] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:13.633494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86df4-6c0c-487f-9919-3fce1f7f53a4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:13.643576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:13.651654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 418/828] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:25:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:13.654422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9028e7-18a5-4d68-918b-7b47ef314de9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:13.654705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9028e7-18a5-4d68-918b-7b47ef314de9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:13.655028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9028e7-18a5-4d68-918b-7b47ef314de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:13.683106 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8f9028e7-18a5-4d68-918b-7b47ef314de9 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:13.685008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8f9028e7-18a5-4d68-918b-7b47ef314de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:13.778354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed86df4-6c0c-487f-9919-3fce1f7f53a4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:13.787296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 419/829] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:13.819155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd68cc-2763-473c-94b4-5b6c45cb841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:13.819851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd68cc-2763-473c-94b4-5b6c45cb841d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:13.820134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd68cc-2763-473c-94b4-5b6c45cb841d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:13.820407 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd68cc-2763-473c-94b4-5b6c45cb841d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:13.833073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd68cc-2763-473c-94b4-5b6c45cb841d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:13.834014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd68cc-2763-473c-94b4-5b6c45cb841d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-8bcd68cc-2763-473c-94b4-5b6c45cb841d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:13.852744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:13.853753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:13.854756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:13.854756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:13.854756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:13.854756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 411/830] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:13.863277 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9028e7-18a5-4d68-918b-7b47ef314de9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:13.902092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:13.973670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9028e7-18a5-4d68-918b-7b47ef314de9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:14.008628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047a532-89c5-49a6-9bfb-934f55276896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:14.009258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047a532-89c5-49a6-9bfb-934f55276896 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:14.009534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047a532-89c5-49a6-9bfb-934f55276896 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:14.009804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047a532-89c5-49a6-9bfb-934f55276896 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:14.010502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047a532-89c5-49a6-9bfb-934f55276896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:14.010876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047a532-89c5-49a6-9bfb-934f55276896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:14.012355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047a532-89c5-49a6-9bfb-934f55276896 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:14.026643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 420/831] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:14.032238 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd68cc-2763-473c-94b4-5b6c45cb841d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:14.065732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047a532-89c5-49a6-9bfb-934f55276896 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:14.097075 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:14.143436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd68cc-2763-473c-94b4-5b6c45cb841d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:14.164447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:14.171659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047a532-89c5-49a6-9bfb-934f55276896 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:14.181675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5eff78-e75d-4f39-8ad5-2529027ef26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:14.182451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5eff78-e75d-4f39-8ad5-2529027ef26a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:14.182631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5eff78-e75d-4f39-8ad5-2529027ef26a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:14.182867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5eff78-e75d-4f39-8ad5-2529027ef26a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:14.191167 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5eff78-e75d-4f39-8ad5-2529027ef26a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:14.192861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5eff78-e75d-4f39-8ad5-2529027ef26a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-df5eff78-e75d-4f39-8ad5-2529027ef26a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:14.205141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:14.206027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:14.206950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:14.206950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:14.206950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:14.206950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:14.206950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:14.206950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:14.206950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:14.206950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:14.211284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 412/832] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:14.212054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5eff78-e75d-4f39-8ad5-2529027ef26a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:14.212834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51624722-b60d-44e3-96e2-fbbe8455fe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:14.213697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51624722-b60d-44e3-96e2-fbbe8455fe97 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:14.214209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51624722-b60d-44e3-96e2-fbbe8455fe97 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:14.214716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51624722-b60d-44e3-96e2-fbbe8455fe97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:14.225749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51624722-b60d-44e3-96e2-fbbe8455fe97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-51624722-b60d-44e3-96e2-fbbe8455fe97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:14.233714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:14.235240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:25:14.236079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:14.238768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 421/833] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:25:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:14.244487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51624722-b60d-44e3-96e2-fbbe8455fe97 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:14.335854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5eff78-e75d-4f39-8ad5-2529027ef26a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:14.350997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:14.358663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51624722-b60d-44e3-96e2-fbbe8455fe97 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:14.374502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314990ef-7458-44e3-9560-5a581dad77be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:14.376042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314990ef-7458-44e3-9560-5a581dad77be None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:14.376317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314990ef-7458-44e3-9560-5a581dad77be None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:14.376628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314990ef-7458-44e3-9560-5a581dad77be None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:14.377265 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314990ef-7458-44e3-9560-5a581dad77be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:14.377565 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314990ef-7458-44e3-9560-5a581dad77be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:14.379149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314990ef-7458-44e3-9560-5a581dad77be None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:14.394261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 413/834] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:14.399362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314990ef-7458-44e3-9560-5a581dad77be None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:14.403891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c076c2-8974-4627-8c45-1d314123a0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:14.404487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c076c2-8974-4627-8c45-1d314123a0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:14.404842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c076c2-8974-4627-8c45-1d314123a0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:14.405121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c076c2-8974-4627-8c45-1d314123a0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:14.405787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c076c2-8974-4627-8c45-1d314123a0c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:14.406089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c076c2-8974-4627-8c45-1d314123a0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:14.407833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c076c2-8974-4627-8c45-1d314123a0c1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:14.416931 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 422/835] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:25:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:14.422813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c076c2-8974-4627-8c45-1d314123a0c1 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:14.443339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:14.513809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314990ef-7458-44e3-9560-5a581dad77be None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:14.532364 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c076c2-8974-4627-8c45-1d314123a0c1 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:14.551120 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c1fa2-53e8-4ff6-a433-4e35e5f6d108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:14.551715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c1fa2-53e8-4ff6-a433-4e35e5f6d108 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:14.551996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c1fa2-53e8-4ff6-a433-4e35e5f6d108 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:14.552268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c1fa2-53e8-4ff6-a433-4e35e5f6d108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:14.553090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c1fa2-53e8-4ff6-a433-4e35e5f6d108 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=fba4f0473f4c4d3bb2d50b09f3de9837) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:14.553403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c1fa2-53e8-4ff6-a433-4e35e5f6d108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:14.554852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c1fa2-53e8-4ff6-a433-4e35e5f6d108 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:14.566583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 414/836] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:14 2022] GET /identity/v3/role_assignments?user.id=fba4f0473f4c4d3bb2d50b09f3de9837&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:14.572466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23875fb-c69a-4dcd-b8f0-e522a61cbe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:14.573519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23875fb-c69a-4dcd-b8f0-e522a61cbe45 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:14.573994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23875fb-c69a-4dcd-b8f0-e522a61cbe45 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:14.574495 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23875fb-c69a-4dcd-b8f0-e522a61cbe45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:14.586839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23875fb-c69a-4dcd-b8f0-e522a61cbe45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:14.588696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23875fb-c69a-4dcd-b8f0-e522a61cbe45 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a23875fb-c69a-4dcd-b8f0-e522a61cbe45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:14.605955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:14.606973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:14.607876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:14.607876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:14.607876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:14.610660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 423/837] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:14.622202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c1fa2-53e8-4ff6-a433-4e35e5f6d108 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:14.667157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:14.736415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c1fa2-53e8-4ff6-a433-4e35e5f6d108 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:14.771357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf1e89-f6e3-4763-b7cc-763661104a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:14.771964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf1e89-f6e3-4763-b7cc-763661104a8a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:14.772302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf1e89-f6e3-4763-b7cc-763661104a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:14.772576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf1e89-f6e3-4763-b7cc-763661104a8a None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:14.773214 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf1e89-f6e3-4763-b7cc-763661104a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:14.773513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf1e89-f6e3-4763-b7cc-763661104a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:14.774903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf1e89-f6e3-4763-b7cc-763661104a8a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:14.787815 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 415/838] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:14.793307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23875fb-c69a-4dcd-b8f0-e522a61cbe45 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:14.867411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:14.902321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23875fb-c69a-4dcd-b8f0-e522a61cbe45 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:14.937903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f93add-335f-4f72-8bb1-e181a6cb64e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:14.938715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f93add-335f-4f72-8bb1-e181a6cb64e5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:14.938998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f93add-335f-4f72-8bb1-e181a6cb64e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:14.939432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f93add-335f-4f72-8bb1-e181a6cb64e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:14.949111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f93add-335f-4f72-8bb1-e181a6cb64e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:14.950580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f93add-335f-4f72-8bb1-e181a6cb64e5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-24f93add-335f-4f72-8bb1-e181a6cb64e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:14.962979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:14.963927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:14.964857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 424/839] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 05:25:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:14.972758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf1e89-f6e3-4763-b7cc-763661104a8a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:15.076513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:15.080934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf1e89-f6e3-4763-b7cc-763661104a8a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:15.116244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2148817-9043-4e21-b6b1-c78e39177cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:15.116853 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2148817-9043-4e21-b6b1-c78e39177cde None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:15.117128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2148817-9043-4e21-b6b1-c78e39177cde None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:15.117400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2148817-9043-4e21-b6b1-c78e39177cde None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:15.118124 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2148817-9043-4e21-b6b1-c78e39177cde None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:15.118460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2148817-9043-4e21-b6b1-c78e39177cde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:15.119866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2148817-9043-4e21-b6b1-c78e39177cde None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:15.137976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 416/840] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 05:25:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:15.144075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f93add-335f-4f72-8bb1-e181a6cb64e5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:15.255204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f93add-335f-4f72-8bb1-e181a6cb64e5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:15.282583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:15.290699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b724870-abf5-441c-a856-a857cd67f16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:15.291316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b724870-abf5-441c-a856-a857cd67f16d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:15.291638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b724870-abf5-441c-a856-a857cd67f16d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:15.291909 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b724870-abf5-441c-a856-a857cd67f16d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:15.331522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f53d65-1d97-44ce-9356-a051ce0106d3 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:15.332129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f53d65-1d97-44ce-9356-a051ce0106d3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:15.332600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f53d65-1d97-44ce-9356-a051ce0106d3 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:15.334378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 417/841] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:15.340223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d2262-5fa8-480a-bfec-e4a2131b8461 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:15.340782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d2262-5fa8-480a-bfec-e4a2131b8461 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:15.341494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d2262-5fa8-480a-bfec-e4a2131b8461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:15.361868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-910d2262-5fa8-480a-bfec-e4a2131b8461 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:15.363080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-910d2262-5fa8-480a-bfec-e4a2131b8461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:15.367471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b724870-abf5-441c-a856-a857cd67f16d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:15.369193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b724870-abf5-441c-a856-a857cd67f16d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:15.461057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 418/842] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:15.478484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 425/843] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:25:15.487195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d2262-5fa8-480a-bfec-e4a2131b8461 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:15.575187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:15.591657 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d2262-5fa8-480a-bfec-e4a2131b8461 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:15.628836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ab54f-af77-4978-96fa-3912da90b7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:15.629472 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ab54f-af77-4978-96fa-3912da90b7ca None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:15.629748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ab54f-af77-4978-96fa-3912da90b7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:15.630100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ab54f-af77-4978-96fa-3912da90b7ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:15.640163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ab54f-af77-4978-96fa-3912da90b7ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:15.641626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ab54f-af77-4978-96fa-3912da90b7ca None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-297ab54f-af77-4978-96fa-3912da90b7ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:15.655980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:15.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:15.657872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:15.657872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:15.657872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:15.657872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 419/844] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:15.664142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b724870-abf5-441c-a856-a857cd67f16d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:15.693443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:15.743904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ab54f-af77-4978-96fa-3912da90b7ca None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:15.769307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b724870-abf5-441c-a856-a857cd67f16d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:15.805700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a148b8-44fb-466e-b766-14458ef6ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:15.807233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a148b8-44fb-466e-b766-14458ef6ffe0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:15.807704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a148b8-44fb-466e-b766-14458ef6ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:15.808163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a148b8-44fb-466e-b766-14458ef6ffe0 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:15.809114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a148b8-44fb-466e-b766-14458ef6ffe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:15.809608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a148b8-44fb-466e-b766-14458ef6ffe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:15.811204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a148b8-44fb-466e-b766-14458ef6ffe0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:15.829533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 426/845] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:15.830760 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a148b8-44fb-466e-b766-14458ef6ffe0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:15.866615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ab54f-af77-4978-96fa-3912da90b7ca None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:15.928662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad9d85-92fe-45b1-ab1f-13aa6844defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:15.929470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9d85-92fe-45b1-ab1f-13aa6844defe None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:15.929721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9d85-92fe-45b1-ab1f-13aa6844defe None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:15.929986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9d85-92fe-45b1-ab1f-13aa6844defe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:15.934286 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:15.939492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:15.945851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad9d85-92fe-45b1-ab1f-13aa6844defe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:15.948673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a148b8-44fb-466e-b766-14458ef6ffe0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-bbad9d85-92fe-45b1-ab1f-13aa6844defe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:15.951982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:15.952914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:15.953782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 420/846] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:15.960164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad9d85-92fe-45b1-ab1f-13aa6844defe None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:15.991003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ee27c-2274-478f-a41d-04d542c5de19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:15.991960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ee27c-2274-478f-a41d-04d542c5de19 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:15.992437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ee27c-2274-478f-a41d-04d542c5de19 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:15.993124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ee27c-2274-478f-a41d-04d542c5de19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.000822 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ee27c-2274-478f-a41d-04d542c5de19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:16.002296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ee27c-2274-478f-a41d-04d542c5de19 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ab6ee27c-2274-478f-a41d-04d542c5de19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:16.013466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:16.014405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:16.015313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:16.015313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:16.015313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:16.015313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:16.015313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:16.015313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:16.015313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:16.015313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:16.015313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 427/847] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:16.016356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d676ab-eed0-4979-bb5d-655548f4fbe5 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.016356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d676ab-eed0-4979-bb5d-655548f4fbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.016356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d676ab-eed0-4979-bb5d-655548f4fbe5 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.017757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 428/848] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:16.022454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d676ab-eed0-4979-bb5d-655548f4fbe5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:16.080976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad9d85-92fe-45b1-ab1f-13aa6844defe None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:16.117159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77539689-896d-42b0-aa4c-e18fee6da07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:16.117763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77539689-896d-42b0-aa4c-e18fee6da07d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.118038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77539689-896d-42b0-aa4c-e18fee6da07d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.118600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77539689-896d-42b0-aa4c-e18fee6da07d None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.118977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77539689-896d-42b0-aa4c-e18fee6da07d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:16.119275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77539689-896d-42b0-aa4c-e18fee6da07d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:16.120644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77539689-896d-42b0-aa4c-e18fee6da07d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:16.130245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 421/849] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:25:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:16.131998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d676ab-eed0-4979-bb5d-655548f4fbe5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:16.136278 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca344fad-345b-4d28-a87a-6493b3ced277 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.136779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca344fad-345b-4d28-a87a-6493b3ced277 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.137264 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca344fad-345b-4d28-a87a-6493b3ced277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.157601 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ca344fad-345b-4d28-a87a-6493b3ced277 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:16.158797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca344fad-345b-4d28-a87a-6493b3ced277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:16.166485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a1030-bc3c-415b-8670-7cfc50e34f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:16.167105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a1030-bc3c-415b-8670-7cfc50e34f4b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.167382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a1030-bc3c-415b-8670-7cfc50e34f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.167650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a1030-bc3c-415b-8670-7cfc50e34f4b None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.168330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a1030-bc3c-415b-8670-7cfc50e34f4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:16.168628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a1030-bc3c-415b-8670-7cfc50e34f4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:16.170031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a1030-bc3c-415b-8670-7cfc50e34f4b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:16.187583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 429/850] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:16.187583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a1030-bc3c-415b-8670-7cfc50e34f4b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:16.199070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:16.223057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:16.237916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 422/851] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:16.239205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca344fad-345b-4d28-a87a-6493b3ced277 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:16.291314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a1030-bc3c-415b-8670-7cfc50e34f4b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:16.326386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5e4b-a5b8-46c3-bea5-c7b1e976be57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:16.327025 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5e4b-a5b8-46c3-bea5-c7b1e976be57 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.327302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5e4b-a5b8-46c3-bea5-c7b1e976be57 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.327572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5e4b-a5b8-46c3-bea5-c7b1e976be57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.338745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c5e4b-a5b8-46c3-bea5-c7b1e976be57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:16.340241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c5e4b-a5b8-46c3-bea5-c7b1e976be57 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-746c5e4b-a5b8-46c3-bea5-c7b1e976be57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:16.355943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:16.356923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:16.357833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:16.357833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:16.357833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:16.362573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 430/852] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:16.363660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5e4b-a5b8-46c3-bea5-c7b1e976be57 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:16.369395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca344fad-345b-4d28-a87a-6493b3ced277 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:16.413501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6e444-872a-4f10-83b9-a77844614df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:16.414114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6e444-872a-4f10-83b9-a77844614df6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.414982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6e444-872a-4f10-83b9-a77844614df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.415263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6e444-872a-4f10-83b9-a77844614df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.415987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6e444-872a-4f10-83b9-a77844614df6 None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=5cf157961e3d45d5ad861f9b4f51ab50) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:16.416294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6e444-872a-4f10-83b9-a77844614df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:16.418918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6e444-872a-4f10-83b9-a77844614df6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:16.427077 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 423/853] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:16 2022] GET /identity/v3/role_assignments?user.id=5cf157961e3d45d5ad861f9b4f51ab50&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:16.432435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:16.480084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5e4b-a5b8-46c3-bea5-c7b1e976be57 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:16.509201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6e444-872a-4f10-83b9-a77844614df6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:16.521863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815cb5c-0ceb-4ebc-ba20-f8d81aa1664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:16.522076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815cb5c-0ceb-4ebc-ba20-f8d81aa1664f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.522426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815cb5c-0ceb-4ebc-ba20-f8d81aa1664f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.522700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815cb5c-0ceb-4ebc-ba20-f8d81aa1664f None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.523368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815cb5c-0ceb-4ebc-ba20-f8d81aa1664f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:16.523666 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815cb5c-0ceb-4ebc-ba20-f8d81aa1664f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:16.525077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815cb5c-0ceb-4ebc-ba20-f8d81aa1664f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:16.539085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 431/854] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:16.547812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815cb5c-0ceb-4ebc-ba20-f8d81aa1664f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:16.585154 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:16.653817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6e444-872a-4f10-83b9-a77844614df6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:16.681453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815cb5c-0ceb-4ebc-ba20-f8d81aa1664f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:16.696971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e5d6b-955a-4c14-b429-da5a037ba5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:16.698251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e5d6b-955a-4c14-b429-da5a037ba5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.698741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e5d6b-955a-4c14-b429-da5a037ba5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.699441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e5d6b-955a-4c14-b429-da5a037ba5f2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.700612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e5d6b-955a-4c14-b429-da5a037ba5f2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:16.702213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e5d6b-955a-4c14-b429-da5a037ba5f2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:16.703844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-142e5d6b-955a-4c14-b429-da5a037ba5f2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:16.713682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:16.719796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:16.720621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 424/855] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:25:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:16.723128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e5d6b-955a-4c14-b429-da5a037ba5f2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:16.726876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4023e7-1b31-41a7-814c-3b11edef6f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:16.727520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4023e7-1b31-41a7-814c-3b11edef6f44 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.728671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4023e7-1b31-41a7-814c-3b11edef6f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.728960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4023e7-1b31-41a7-814c-3b11edef6f44 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.738714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4023e7-1b31-41a7-814c-3b11edef6f44 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:16.740535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4023e7-1b31-41a7-814c-3b11edef6f44 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9a4023e7-1b31-41a7-814c-3b11edef6f44 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:16.755253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:16.756338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:16.757334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:16.757334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:16.757334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:16.757334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:16.757334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:16.757334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:16.757334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:16.757334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:16.757334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 05:25:16.757334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:16.764298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 432/856] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:16.766473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4023e7-1b31-41a7-814c-3b11edef6f44 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:16.861923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e5d6b-955a-4c14-b429-da5a037ba5f2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:16.870599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4023e7-1b31-41a7-814c-3b11edef6f44 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:16.886222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:16.907712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bd5be-7461-4d34-9c45-cdb4edacb171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:16.908674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bd5be-7461-4d34-9c45-cdb4edacb171 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.909109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bd5be-7461-4d34-9c45-cdb4edacb171 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.909377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bd5be-7461-4d34-9c45-cdb4edacb171 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.910012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bd5be-7461-4d34-9c45-cdb4edacb171 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:16.910423 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bd5be-7461-4d34-9c45-cdb4edacb171 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:16.911781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bd5be-7461-4d34-9c45-cdb4edacb171 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:16.912118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed51f4e-44a3-49dc-b393-3c4541508a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:16.912842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed51f4e-44a3-49dc-b393-3c4541508a15 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.913149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed51f4e-44a3-49dc-b393-3c4541508a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.913523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed51f4e-44a3-49dc-b393-3c4541508a15 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.914260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed51f4e-44a3-49dc-b393-3c4541508a15 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:16.914563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed51f4e-44a3-49dc-b393-3c4541508a15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:16.915963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed51f4e-44a3-49dc-b393-3c4541508a15 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:16.921410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 425/857] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:25:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:16.921410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa3495-8dfa-4f77-8803-567209876f6b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.921410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa3495-8dfa-4f77-8803-567209876f6b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.921410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa3495-8dfa-4f77-8803-567209876f6b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.921410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 426/858] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:16.922051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa3495-8dfa-4f77-8803-567209876f6b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:16.945932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 433/859] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 05:25:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:16.945932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4942251-6758-4daf-acde-6e41469d93ad None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:16.945932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4942251-6758-4daf-acde-6e41469d93ad None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:16.945932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4942251-6758-4daf-acde-6e41469d93ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:16.962641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e4942251-6758-4daf-acde-6e41469d93ad None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:16.963926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e4942251-6758-4daf-acde-6e41469d93ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:16.993826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:17.037249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 434/860] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:17.038158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4942251-6758-4daf-acde-6e41469d93ad None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:17.052553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa3495-8dfa-4f77-8803-567209876f6b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:17.095512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e98c5-c4b5-4f72-b8e7-b24eb2d58683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:17.096153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e98c5-c4b5-4f72-b8e7-b24eb2d58683 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:17.096429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e98c5-c4b5-4f72-b8e7-b24eb2d58683 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:17.097082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e98c5-c4b5-4f72-b8e7-b24eb2d58683 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:17.097721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e98c5-c4b5-4f72-b8e7-b24eb2d58683 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:17.098023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e98c5-c4b5-4f72-b8e7-b24eb2d58683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:17.099435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e98c5-c4b5-4f72-b8e7-b24eb2d58683 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:17.110323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 427/861] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:25:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:17.166396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4942251-6758-4daf-acde-6e41469d93ad None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:17.209132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf2828-50ae-4e81-bf28-1ea424d58a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:17.209753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf2828-50ae-4e81-bf28-1ea424d58a20 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:17.210030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf2828-50ae-4e81-bf28-1ea424d58a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:17.210331 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf2828-50ae-4e81-bf28-1ea424d58a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:17.250772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:17.288967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf2828-50ae-4e81-bf28-1ea424d58a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:17.292652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf2828-50ae-4e81-bf28-1ea424d58a20 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:17.315814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e98c5-c4b5-4f72-b8e7-b24eb2d58683 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:17.389414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:17.410300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 435/862] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:25:17.414239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf2828-50ae-4e81-bf28-1ea424d58a20 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:17.459728 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e98c5-c4b5-4f72-b8e7-b24eb2d58683 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:17.486685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:17.504439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93643a8-1cf7-4c6d-a7b8-0729d2aaf3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:17.505435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93643a8-1cf7-4c6d-a7b8-0729d2aaf3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:17.505718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93643a8-1cf7-4c6d-a7b8-0729d2aaf3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:17.505995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93643a8-1cf7-4c6d-a7b8-0729d2aaf3d7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:17.506747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93643a8-1cf7-4c6d-a7b8-0729d2aaf3d7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:17.508397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93643a8-1cf7-4c6d-a7b8-0729d2aaf3d7 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d93643a8-1cf7-4c6d-a7b8-0729d2aaf3d7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:17.519467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:17.520456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 05:25:17.521343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:17.525674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 428/863] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Oct 4 05:25:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:17.526795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93643a8-1cf7-4c6d-a7b8-0729d2aaf3d7 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:17.532267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf2828-50ae-4e81-bf28-1ea424d58a20 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:17.567046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a470ec5-373d-4a54-b3bc-2645455fbd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:17.568068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a470ec5-373d-4a54-b3bc-2645455fbd58 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:17.568551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a470ec5-373d-4a54-b3bc-2645455fbd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:17.569306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a470ec5-373d-4a54-b3bc-2645455fbd58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:17.580540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a470ec5-373d-4a54-b3bc-2645455fbd58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:17.582450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a470ec5-373d-4a54-b3bc-2645455fbd58 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9a470ec5-373d-4a54-b3bc-2645455fbd58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:17.597831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:17.598815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:17.599705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:17.599705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:17.599705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:17.606209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 436/864] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:17.607709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a470ec5-373d-4a54-b3bc-2645455fbd58 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:17.645833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93643a8-1cf7-4c6d-a7b8-0729d2aaf3d7 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:17.660371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:17.689510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bd382-d471-4577-a4d4-f08d3261aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:17.690406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bd382-d471-4577-a4d4-f08d3261aa5a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:17.690836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bd382-d471-4577-a4d4-f08d3261aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:17.691327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bd382-d471-4577-a4d4-f08d3261aa5a None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:17.692243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bd382-d471-4577-a4d4-f08d3261aa5a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:17.692749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bd382-d471-4577-a4d4-f08d3261aa5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:17.694827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9bd382-d471-4577-a4d4-f08d3261aa5a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:17.700222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 429/865] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:17.705598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bd382-d471-4577-a4d4-f08d3261aa5a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:17.729937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a470ec5-373d-4a54-b3bc-2645455fbd58 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:17.752288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:17.768244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d302aa-eb86-435d-9480-c8fbe99a9e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:17.768866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d302aa-eb86-435d-9480-c8fbe99a9e03 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:17.769146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d302aa-eb86-435d-9480-c8fbe99a9e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:17.769419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d302aa-eb86-435d-9480-c8fbe99a9e03 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:17.770065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d302aa-eb86-435d-9480-c8fbe99a9e03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:17.770417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d302aa-eb86-435d-9480-c8fbe99a9e03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:17.771961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d302aa-eb86-435d-9480-c8fbe99a9e03 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:17.791910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 437/866] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:17.792746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d302aa-eb86-435d-9480-c8fbe99a9e03 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:17.827706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bd382-d471-4577-a4d4-f08d3261aa5a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:17.870700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2a1dc-0040-43df-9f26-7ce46f9fb35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:17.871953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2a1dc-0040-43df-9f26-7ce46f9fb35d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:17.872849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2a1dc-0040-43df-9f26-7ce46f9fb35d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:17.873594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2a1dc-0040-43df-9f26-7ce46f9fb35d None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:17.874465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2a1dc-0040-43df-9f26-7ce46f9fb35d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:17.875015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2a1dc-0040-43df-9f26-7ce46f9fb35d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:17.876841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2a1dc-0040-43df-9f26-7ce46f9fb35d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:17.891276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 430/867] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:17.931273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d302aa-eb86-435d-9480-c8fbe99a9e03 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:17.971443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585523d2-c51a-40c7-9497-20cf9210c6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:17.972455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585523d2-c51a-40c7-9497-20cf9210c6be None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:17.973064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585523d2-c51a-40c7-9497-20cf9210c6be None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:17.973564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585523d2-c51a-40c7-9497-20cf9210c6be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:17.981731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585523d2-c51a-40c7-9497-20cf9210c6be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:17.983687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585523d2-c51a-40c7-9497-20cf9210c6be None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-585523d2-c51a-40c7-9497-20cf9210c6be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:17.996700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:17.997700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:17.998668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:17.998668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:17.998668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:17.998668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:17.998668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:17.998668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:17.998668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:17.998668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:18.000915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 438/868] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:18.006763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2a1dc-0040-43df-9f26-7ce46f9fb35d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:18.056672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:18.124258 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2a1dc-0040-43df-9f26-7ce46f9fb35d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:18.160646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e85ee-f698-49e5-9ea8-70da3da2c5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:18.161401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e85ee-f698-49e5-9ea8-70da3da2c5ee None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:18.161775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e85ee-f698-49e5-9ea8-70da3da2c5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:18.162184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e85ee-f698-49e5-9ea8-70da3da2c5ee None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:18.162968 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e85ee-f698-49e5-9ea8-70da3da2c5ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:18.163366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e85ee-f698-49e5-9ea8-70da3da2c5ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:18.164860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e85ee-f698-49e5-9ea8-70da3da2c5ee None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:18.177123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 431/869] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:18.182923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585523d2-c51a-40c7-9497-20cf9210c6be None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:18.230031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:18.291105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585523d2-c51a-40c7-9497-20cf9210c6be None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:18.327175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0111be1-feaa-454a-9545-bfbb91627109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:18.327794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0111be1-feaa-454a-9545-bfbb91627109 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:18.328603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0111be1-feaa-454a-9545-bfbb91627109 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:18.328885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0111be1-feaa-454a-9545-bfbb91627109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:18.329614 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0111be1-feaa-454a-9545-bfbb91627109 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=783ad0cd9721409e8a7114cecef16aab) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:18.329922 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0111be1-feaa-454a-9545-bfbb91627109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:18.331437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0111be1-feaa-454a-9545-bfbb91627109 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:18.342301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 439/870] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:18 2022] GET /identity/v3/role_assignments?user.id=783ad0cd9721409e8a7114cecef16aab&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:18.745098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc16785-d039-4dd6-af0d-503422a9dcd2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:18.745614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc16785-d039-4dd6-af0d-503422a9dcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:18.745717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc16785-d039-4dd6-af0d-503422a9dcd2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:18.747174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 432/871] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:18.753165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bb532-b71d-40a2-b853-55e8d663d963 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:18.753702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bb532-b71d-40a2-b853-55e8d663d963 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:18.754215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bb532-b71d-40a2-b853-55e8d663d963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:18.772809 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b30bb532-b71d-40a2-b853-55e8d663d963 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:18.773992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b30bb532-b71d-40a2-b853-55e8d663d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:18.847613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 440/872] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:19.118699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc16785-d039-4dd6-af0d-503422a9dcd2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:19.227833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:19.233667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc16785-d039-4dd6-af0d-503422a9dcd2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:19.274433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694cd04-4715-4671-9193-b6ed50c4a4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:19.275518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694cd04-4715-4671-9193-b6ed50c4a4dc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:19.275801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694cd04-4715-4671-9193-b6ed50c4a4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:19.276078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694cd04-4715-4671-9193-b6ed50c4a4dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:19.289443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694cd04-4715-4671-9193-b6ed50c4a4dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0694cd04-4715-4671-9193-b6ed50c4a4dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:19.294397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:19.295329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:19.296197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 433/873] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 05:25:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:19.303433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bb532-b71d-40a2-b853-55e8d663d963 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:19.339226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:19.352924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaeaac-a860-4f8d-9a99-b49cefc98705 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:19.353215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaeaac-a860-4f8d-9a99-b49cefc98705 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:19.353498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaeaac-a860-4f8d-9a99-b49cefc98705 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:19.354997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 434/874] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:19.361062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a3f0f-ac5e-49e1-9080-46e1cfd3497e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:19.361490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a3f0f-ac5e-49e1-9080-46e1cfd3497e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:19.361766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a3f0f-ac5e-49e1-9080-46e1cfd3497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:19.383620 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f96a3f0f-ac5e-49e1-9080-46e1cfd3497e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:19.385037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f96a3f0f-ac5e-49e1-9080-46e1cfd3497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:19.414741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bb532-b71d-40a2-b853-55e8d663d963 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:19.457913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8f588-89e8-48cb-b739-5f8fd643a0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:19.458588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8f588-89e8-48cb-b739-5f8fd643a0ba None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:19.458867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8f588-89e8-48cb-b739-5f8fd643a0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:19.459142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8f588-89e8-48cb-b739-5f8fd643a0ba None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:19.459856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8f588-89e8-48cb-b739-5f8fd643a0ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:19.460173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8f588-89e8-48cb-b739-5f8fd643a0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:19.466170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8f588-89e8-48cb-b739-5f8fd643a0ba None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:19.466311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 435/875] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:19.474084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 441/876] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 05:25:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:19.479814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a3f0f-ac5e-49e1-9080-46e1cfd3497e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:19.571104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:19.582227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a3f0f-ac5e-49e1-9080-46e1cfd3497e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:19.617268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f41fb0-43cf-4844-b468-a3254c794509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:19.617904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f41fb0-43cf-4844-b468-a3254c794509 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:19.618211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f41fb0-43cf-4844-b468-a3254c794509 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:19.618487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f41fb0-43cf-4844-b468-a3254c794509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:19.629383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f41fb0-43cf-4844-b468-a3254c794509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:19.630857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f41fb0-43cf-4844-b468-a3254c794509 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-81f41fb0-43cf-4844-b468-a3254c794509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:19.645737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:19.646755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:19.647702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:19.647702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:19.647702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:19.647702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 436/877] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:19.652678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8f588-89e8-48cb-b739-5f8fd643a0ba None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:19.743947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f41fb0-43cf-4844-b468-a3254c794509 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:19.758369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:19.766290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8f588-89e8-48cb-b739-5f8fd643a0ba None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:19.811988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64879ed-f5b7-448b-95b7-9af107c75e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:19.812635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64879ed-f5b7-448b-95b7-9af107c75e47 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:19.812911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64879ed-f5b7-448b-95b7-9af107c75e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:19.813189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64879ed-f5b7-448b-95b7-9af107c75e47 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:19.813994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64879ed-f5b7-448b-95b7-9af107c75e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:19.814329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64879ed-f5b7-448b-95b7-9af107c75e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:19.816238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64879ed-f5b7-448b-95b7-9af107c75e47 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:19.834964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 442/878] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:19.836459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64879ed-f5b7-448b-95b7-9af107c75e47 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:19.859011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f41fb0-43cf-4844-b468-a3254c794509 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:19.905359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad35fa0-35e8-476d-8066-fe687de88aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:19.906586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad35fa0-35e8-476d-8066-fe687de88aeb None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:19.907018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad35fa0-35e8-476d-8066-fe687de88aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:19.907317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad35fa0-35e8-476d-8066-fe687de88aeb None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:19.907942 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad35fa0-35e8-476d-8066-fe687de88aeb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:19.909965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad35fa0-35e8-476d-8066-fe687de88aeb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:19.954737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64879ed-f5b7-448b-95b7-9af107c75e47 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:19.990009 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93aad8-d615-4b09-9a77-86c714cacf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:19.990665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93aad8-d615-4b09-9a77-86c714cacf1e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:19.990940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93aad8-d615-4b09-9a77-86c714cacf1e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:19.991209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93aad8-d615-4b09-9a77-86c714cacf1e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:19.994944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:20.000306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93aad8-d615-4b09-9a77-86c714cacf1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:20.001469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 437/879] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:25:19 2022] POST /identity/v3/services => generated 251 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:25:20.001790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93aad8-d615-4b09-9a77-86c714cacf1e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9e93aad8-d615-4b09-9a77-86c714cacf1e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:20.019730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:20.020798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:20.021750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:20.021750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:20.021750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:20.021750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:20.021750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:20.021750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:20.021750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:20.021750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:20.021750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:20.021750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:20.024682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 443/880] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 05:25:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:20.030467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad35fa0-35e8-476d-8066-fe687de88aeb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:20.153773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad35fa0-35e8-476d-8066-fe687de88aeb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:20.194181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fbf61-051f-417c-999e-8c978ce4dc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:20.195158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fbf61-051f-417c-999e-8c978ce4dc5c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:20.195666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fbf61-051f-417c-999e-8c978ce4dc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:20.196140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fbf61-051f-417c-999e-8c978ce4dc5c None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:20.197091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fbf61-051f-417c-999e-8c978ce4dc5c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:20.197621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fbf61-051f-417c-999e-8c978ce4dc5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:20.199317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fbf61-051f-417c-999e-8c978ce4dc5c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:20.222171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 438/881] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 05:25:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:20.224938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3d336-f0cf-4b47-8591-94f817b11c27 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:20.225224 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3d336-f0cf-4b47-8591-94f817b11c27 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:20.225502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3d336-f0cf-4b47-8591-94f817b11c27 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:20.227085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 444/882] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:20.228149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fbf61-051f-417c-999e-8c978ce4dc5c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:20.233972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a517591-e569-40b3-88b5-1df94a072d36 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:20.233972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a517591-e569-40b3-88b5-1df94a072d36 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:20.234371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a517591-e569-40b3-88b5-1df94a072d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:20.258782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4a517591-e569-40b3-88b5-1df94a072d36 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:20.260448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4a517591-e569-40b3-88b5-1df94a072d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:20.275687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:20.295627 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:20.352150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fbf61-051f-417c-999e-8c978ce4dc5c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:20.354108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 445/883] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:20.386993 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88076727-5549-41fc-a994-09ba14ec188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:20.387607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076727-5549-41fc-a994-09ba14ec188a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:20.387887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076727-5549-41fc-a994-09ba14ec188a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:20.388213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88076727-5549-41fc-a994-09ba14ec188a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:20.458354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88076727-5549-41fc-a994-09ba14ec188a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:20.460180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88076727-5549-41fc-a994-09ba14ec188a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:20.522506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:20.567091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 439/884] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:20 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 05:25:20.572625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a517591-e569-40b3-88b5-1df94a072d36 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:20.598090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500097a-df6d-4792-8e0d-b639ec0abde6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:20.598548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500097a-df6d-4792-8e0d-b639ec0abde6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:20.598827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500097a-df6d-4792-8e0d-b639ec0abde6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:20.600841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 440/885] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:20.606812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230501a-1d71-47de-bad8-ec295d801342 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:20.607375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230501a-1d71-47de-bad8-ec295d801342 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:20.607659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230501a-1d71-47de-bad8-ec295d801342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:20.630561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a230501a-1d71-47de-bad8-ec295d801342 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:20.632500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a230501a-1d71-47de-bad8-ec295d801342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:20.674616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a517591-e569-40b3-88b5-1df94a072d36 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:20.709936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23554437-efe6-4c52-a6de-985087db42a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:20.710640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23554437-efe6-4c52-a6de-985087db42a8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:20.710929 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23554437-efe6-4c52-a6de-985087db42a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:20.711208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23554437-efe6-4c52-a6de-985087db42a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:20.716340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 441/886] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:20.716982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230501a-1d71-47de-bad8-ec295d801342 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:20.726248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23554437-efe6-4c52-a6de-985087db42a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:20.728803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23554437-efe6-4c52-a6de-985087db42a8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:20.737887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-23554437-efe6-4c52-a6de-985087db42a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:20.746669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:20.747611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:20.748503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:20.748503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:20.748503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:20.753067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 446/887] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:20.753975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23554437-efe6-4c52-a6de-985087db42a8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:20.830060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:20.852445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230501a-1d71-47de-bad8-ec295d801342 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:20.859039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23554437-efe6-4c52-a6de-985087db42a8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:20.904018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1268e-87bd-4976-9262-d9c5d35c8f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:20.905277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1268e-87bd-4976-9262-d9c5d35c8f3e None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:20.905697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1268e-87bd-4976-9262-d9c5d35c8f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:20.906125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cbb82-1b57-442c-84fc-44aa523a5eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:20.906296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1268e-87bd-4976-9262-d9c5d35c8f3e None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:20.906926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cbb82-1b57-442c-84fc-44aa523a5eda None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:20.907243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1268e-87bd-4976-9262-d9c5d35c8f3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:20.907353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cbb82-1b57-442c-84fc-44aa523a5eda None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:20.907716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cbb82-1b57-442c-84fc-44aa523a5eda None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:20.908463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cbb82-1b57-442c-84fc-44aa523a5eda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:20.908861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cbb82-1b57-442c-84fc-44aa523a5eda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:20.909356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1268e-87bd-4976-9262-d9c5d35c8f3e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:20.910473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cbb82-1b57-442c-84fc-44aa523a5eda None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:20.924403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 447/888] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:20.929781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cbb82-1b57-442c-84fc-44aa523a5eda None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:20.983109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 442/889] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:25:20 2022] POST /identity/v3/services => generated 269 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:20.990974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:20.991303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1268e-87bd-4976-9262-d9c5d35c8f3e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:21.053716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cbb82-1b57-442c-84fc-44aa523a5eda None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:21.113291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2a698-01ce-45eb-b96b-5322044a0dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:21.114168 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a698-01ce-45eb-b96b-5322044a0dbc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:21.114887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a698-01ce-45eb-b96b-5322044a0dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:21.115170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a698-01ce-45eb-b96b-5322044a0dbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:21.124051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1268e-87bd-4976-9262-d9c5d35c8f3e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:21.125911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2a698-01ce-45eb-b96b-5322044a0dbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:21.128778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2a698-01ce-45eb-b96b-5322044a0dbc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-68f2a698-01ce-45eb-b96b-5322044a0dbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:21.144213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:21.145372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:21.146475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:21.146475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:21.146475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:21.146475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:21.146475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:21.146475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:21.146475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:21.146475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:21.152333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 448/890] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:21.154651 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2a698-01ce-45eb-b96b-5322044a0dbc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:21.177395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78a7af-4d39-4c26-9960-515c7c044a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:21.178324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78a7af-4d39-4c26-9960-515c7c044a6e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:21.178608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78a7af-4d39-4c26-9960-515c7c044a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:21.178877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78a7af-4d39-4c26-9960-515c7c044a6e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:21.179464 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78a7af-4d39-4c26-9960-515c7c044a6e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:21.180818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f78a7af-4d39-4c26-9960-515c7c044a6e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2f78a7af-4d39-4c26-9960-515c7c044a6e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:21.191856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:21.192854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:21.193726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:21.194805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 443/891] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:21.201436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78a7af-4d39-4c26-9960-515c7c044a6e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:21.212658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:21.281741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2a698-01ce-45eb-b96b-5322044a0dbc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:21.297300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:21.318027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928f3ed-9181-4427-bf08-6dd4ebca3ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:21.318271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78a7af-4d39-4c26-9960-515c7c044a6e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:21.319574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928f3ed-9181-4427-bf08-6dd4ebca3ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:21.320054 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928f3ed-9181-4427-bf08-6dd4ebca3ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:21.320538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928f3ed-9181-4427-bf08-6dd4ebca3ba5 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:21.321439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928f3ed-9181-4427-bf08-6dd4ebca3ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:21.321941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928f3ed-9181-4427-bf08-6dd4ebca3ba5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:21.323619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928f3ed-9181-4427-bf08-6dd4ebca3ba5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:21.336861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 449/892] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:21.342828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928f3ed-9181-4427-bf08-6dd4ebca3ba5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:21.356798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc043acb-6910-4c3d-9828-4ca51657840c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:21.357416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc043acb-6910-4c3d-9828-4ca51657840c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:21.357741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc043acb-6910-4c3d-9828-4ca51657840c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:21.358020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc043acb-6910-4c3d-9828-4ca51657840c None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:21.358707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc043acb-6910-4c3d-9828-4ca51657840c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:21.359012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc043acb-6910-4c3d-9828-4ca51657840c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:21.361092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc043acb-6910-4c3d-9828-4ca51657840c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:21.365833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 444/893] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 05:25:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:21.371580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc043acb-6910-4c3d-9828-4ca51657840c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:21.450632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:21.456528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928f3ed-9181-4427-bf08-6dd4ebca3ba5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:21.488316 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc043acb-6910-4c3d-9828-4ca51657840c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:21.500108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223c6b0-337e-4d49-8c55-17dfdbc5d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:21.501274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223c6b0-337e-4d49-8c55-17dfdbc5d84a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:21.501944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223c6b0-337e-4d49-8c55-17dfdbc5d84a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:21.502423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223c6b0-337e-4d49-8c55-17dfdbc5d84a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:21.503536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223c6b0-337e-4d49-8c55-17dfdbc5d84a None admin] RBAC: Adding query filter params (role.id=ad98bf0d9c9240e48f0243790aff940d, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=5cf157961e3d45d5ad861f9b4f51ab50) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:21.503992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223c6b0-337e-4d49-8c55-17dfdbc5d84a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:21.506346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223c6b0-337e-4d49-8c55-17dfdbc5d84a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:21.514056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 450/894] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:21 2022] GET /identity/v3/role_assignments?user.id=5cf157961e3d45d5ad861f9b4f51ab50&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=ad98bf0d9c9240e48f0243790aff940d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:21.529663 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0134c23-15e8-4da2-ba88-aa2e5224c612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:21.530538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0134c23-15e8-4da2-ba88-aa2e5224c612 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:21.531445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0134c23-15e8-4da2-ba88-aa2e5224c612 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:21.531888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0134c23-15e8-4da2-ba88-aa2e5224c612 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:21.532698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0134c23-15e8-4da2-ba88-aa2e5224c612 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:21.533125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0134c23-15e8-4da2-ba88-aa2e5224c612 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:21.534738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0134c23-15e8-4da2-ba88-aa2e5224c612 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:21.540614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 445/895] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 05:25:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:22.357771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca724087-5ac1-4258-8e58-35e75fe3522d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:22.358166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca724087-5ac1-4258-8e58-35e75fe3522d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:22.358344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca724087-5ac1-4258-8e58-35e75fe3522d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:22.360458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 451/896] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:22.365753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14391c-b7f9-416b-bf49-75e5d12594b7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:22.366246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14391c-b7f9-416b-bf49-75e5d12594b7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:22.366336 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14391c-b7f9-416b-bf49-75e5d12594b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:22.384876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5b14391c-b7f9-416b-bf49-75e5d12594b7 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:22.386048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5b14391c-b7f9-416b-bf49-75e5d12594b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:22.463332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 446/897] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:22.735478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca724087-5ac1-4258-8e58-35e75fe3522d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:22.777604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:22.840119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca724087-5ac1-4258-8e58-35e75fe3522d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:22.875034 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d984b-1e06-44a4-b7fa-0567dde685d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:22.875672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d984b-1e06-44a4-b7fa-0567dde685d3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:22.875948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d984b-1e06-44a4-b7fa-0567dde685d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:22.876218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d984b-1e06-44a4-b7fa-0567dde685d3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:22.876798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d984b-1e06-44a4-b7fa-0567dde685d3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:22.878165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d984b-1e06-44a4-b7fa-0567dde685d3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a47d984b-1e06-44a4-b7fa-0567dde685d3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:22.888407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:22.889505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:22.890484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 452/898] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:25:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:22.898410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14391c-b7f9-416b-bf49-75e5d12594b7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:22.968982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:23.003949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14391c-b7f9-416b-bf49-75e5d12594b7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:23.038355 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db735a40-097a-419e-9ed4-4c2bd0b2c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:23.039208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db735a40-097a-419e-9ed4-4c2bd0b2c25f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.039642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db735a40-097a-419e-9ed4-4c2bd0b2c25f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.040063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db735a40-097a-419e-9ed4-4c2bd0b2c25f None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.040806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db735a40-097a-419e-9ed4-4c2bd0b2c25f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:23.041271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db735a40-097a-419e-9ed4-4c2bd0b2c25f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:23.042782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db735a40-097a-419e-9ed4-4c2bd0b2c25f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:23.048420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 447/899] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:25:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:23.054100 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d984b-1e06-44a4-b7fa-0567dde685d3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:23.154413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d984b-1e06-44a4-b7fa-0567dde685d3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:23.156860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:23.190813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d795f-2591-499b-9540-479c9fc8c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:23.191443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d795f-2591-499b-9540-479c9fc8c382 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.191718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d795f-2591-499b-9540-479c9fc8c382 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.191988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d795f-2591-499b-9540-479c9fc8c382 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.192633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d795f-2591-499b-9540-479c9fc8c382 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:23.192936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d795f-2591-499b-9540-479c9fc8c382 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:23.194643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d795f-2591-499b-9540-479c9fc8c382 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:23.200878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 453/900] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:25:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:23.207580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db735a40-097a-419e-9ed4-4c2bd0b2c25f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:23.315775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db735a40-097a-419e-9ed4-4c2bd0b2c25f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:23.322527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9b7d7-9b26-4bd1-a6ca-c33fd72677d1 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.322807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9b7d7-9b26-4bd1-a6ca-c33fd72677d1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.323079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9b7d7-9b26-4bd1-a6ca-c33fd72677d1 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.324650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 454/901] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:23.335055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c24f2-02ef-4fd5-b3e4-2f8a24d0f63e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.335650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c24f2-02ef-4fd5-b3e4-2f8a24d0f63e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.336031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c24f2-02ef-4fd5-b3e4-2f8a24d0f63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.356779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:23.359296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1e8c24f2-02ef-4fd5-b3e4-2f8a24d0f63e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:23.360606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1e8c24f2-02ef-4fd5-b3e4-2f8a24d0f63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:23.362796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98458978-25f3-4e00-a480-1c9a0d7211cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:23.363409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458978-25f3-4e00-a480-1c9a0d7211cd None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.363686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458978-25f3-4e00-a480-1c9a0d7211cd None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.363999 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458978-25f3-4e00-a480-1c9a0d7211cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.364664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98458978-25f3-4e00-a480-1c9a0d7211cd None admin] RBAC: Adding query filter params (interface=public, service_id=3db0659555924d3f86a655f1ca33d5ec, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:23.364972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98458978-25f3-4e00-a480-1c9a0d7211cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:23.366339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98458978-25f3-4e00-a480-1c9a0d7211cd None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:23.376583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 448/902] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 05:25:23 2022] GET /identity/v3/endpoints?service_id=3db0659555924d3f86a655f1ca33d5ec&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:23.378281 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98458978-25f3-4e00-a480-1c9a0d7211cd None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:23.440223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 455/903] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:23.492791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98458978-25f3-4e00-a480-1c9a0d7211cd None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:23.527168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46ff80-fd85-4c26-85d7-06b58a33330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:23.527775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46ff80-fd85-4c26-85d7-06b58a33330a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.528048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46ff80-fd85-4c26-85d7-06b58a33330a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.528319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46ff80-fd85-4c26-85d7-06b58a33330a None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.528920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46ff80-fd85-4c26-85d7-06b58a33330a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:23.530292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46ff80-fd85-4c26-85d7-06b58a33330a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:23.537185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 449/904] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 05:25:23 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:25:23.699050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b19f6-4c6b-4957-8c23-e395ae93cbab None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.699549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b19f6-4c6b-4957-8c23-e395ae93cbab None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.699939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b19f6-4c6b-4957-8c23-e395ae93cbab None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.701807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 456/905] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:23.708001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844eaef-81fb-4951-a754-c66e38ba86d6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.708287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844eaef-81fb-4951-a754-c66e38ba86d6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.708555 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844eaef-81fb-4951-a754-c66e38ba86d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.727342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1844eaef-81fb-4951-a754-c66e38ba86d6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:23.728536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1844eaef-81fb-4951-a754-c66e38ba86d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:23.738819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b19f6-4c6b-4957-8c23-e395ae93cbab None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:23.789829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:23.811225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 450/906] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:23.813064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8045cd7-2377-4021-968a-709702adc2bd None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.813530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8045cd7-2377-4021-968a-709702adc2bd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.813773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8045cd7-2377-4021-968a-709702adc2bd None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.818188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 451/907] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:23.820559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca4a98-4b1e-48b3-a2e4-b869d023253a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.821030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca4a98-4b1e-48b3-a2e4-b869d023253a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.821473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca4a98-4b1e-48b3-a2e4-b869d023253a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.847645 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1fca4a98-4b1e-48b3-a2e4-b869d023253a None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:23.849344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1fca4a98-4b1e-48b3-a2e4-b869d023253a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:23.856463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:23.863904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b19f6-4c6b-4957-8c23-e395ae93cbab None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:23.907708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430a8e1-e85c-4e5f-9027-54d85e6e4417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:23.908656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a8e1-e85c-4e5f-9027-54d85e6e4417 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:23.909115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a8e1-e85c-4e5f-9027-54d85e6e4417 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:23.909826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a8e1-e85c-4e5f-9027-54d85e6e4417 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:23.910742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430a8e1-e85c-4e5f-9027-54d85e6e4417 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:23.912311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430a8e1-e85c-4e5f-9027-54d85e6e4417 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9430a8e1-e85c-4e5f-9027-54d85e6e4417 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:23.922484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:23.923453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 05:25:23.924336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:23.927175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 457/908] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Oct 4 05:25:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:23.934191 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430a8e1-e85c-4e5f-9027-54d85e6e4417 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:23.948822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 452/909] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:24.041472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430a8e1-e85c-4e5f-9027-54d85e6e4417 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:24.076230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6d20a-530b-4416-ac1b-1b43ace02696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:24.077157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6d20a-530b-4416-ac1b-1b43ace02696 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:24.077606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6d20a-530b-4416-ac1b-1b43ace02696 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:24.078086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6d20a-530b-4416-ac1b-1b43ace02696 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:24.078913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6d20a-530b-4416-ac1b-1b43ace02696 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:24.079384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6d20a-530b-4416-ac1b-1b43ace02696 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:24.081000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6d20a-530b-4416-ac1b-1b43ace02696 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:24.083345 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca4a98-4b1e-48b3-a2e4-b869d023253a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:24.091643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 458/910] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:24.096834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6d20a-530b-4416-ac1b-1b43ace02696 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:24.147176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:24.189030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca4a98-4b1e-48b3-a2e4-b869d023253a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:24.205317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6d20a-530b-4416-ac1b-1b43ace02696 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:24.207518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:24.225815 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4f289-e41e-49db-b478-b8ed1ca9d55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:24.226482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f289-e41e-49db-b478-b8ed1ca9d55c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:24.226757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f289-e41e-49db-b478-b8ed1ca9d55c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:24.227027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f289-e41e-49db-b478-b8ed1ca9d55c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:24.241355 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4f289-e41e-49db-b478-b8ed1ca9d55c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:24.246690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a9f2e-1481-429a-917d-1d9c743429c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:24.247303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a9f2e-1481-429a-917d-1d9c743429c6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:24.247580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a9f2e-1481-429a-917d-1d9c743429c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1ea4f289-e41e-49db-b478-b8ed1ca9d55c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:24.247789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:24.248682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:24.250731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:24.251735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a9f2e-1481-429a-917d-1d9c743429c6 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:24.251735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a9f2e-1481-429a-917d-1d9c743429c6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:24.251735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a9f2e-1481-429a-917d-1d9c743429c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:24.252225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 453/911] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 05:25:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:24.252225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4f289-e41e-49db-b478-b8ed1ca9d55c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:24.252368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a9f2e-1481-429a-917d-1d9c743429c6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:24.260945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 459/912] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:24.260945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a9f2e-1481-429a-917d-1d9c743429c6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:24.353578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:24.360607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4f289-e41e-49db-b478-b8ed1ca9d55c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:24.366419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a9f2e-1481-429a-917d-1d9c743429c6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:24.396119 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135b216-4e86-4105-981f-416ca043f184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:24.396946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135b216-4e86-4105-981f-416ca043f184 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:24.397155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135b216-4e86-4105-981f-416ca043f184 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:24.397489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135b216-4e86-4105-981f-416ca043f184 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:24.398167 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135b216-4e86-4105-981f-416ca043f184 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:24.399503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135b216-4e86-4105-981f-416ca043f184 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:24.401834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be093e8-20d5-429f-aeb5-7457d41eea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:24.402544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be093e8-20d5-429f-aeb5-7457d41eea6c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:24.402937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be093e8-20d5-429f-aeb5-7457d41eea6c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:24.403280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be093e8-20d5-429f-aeb5-7457d41eea6c None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:24.404021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be093e8-20d5-429f-aeb5-7457d41eea6c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:24.404324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be093e8-20d5-429f-aeb5-7457d41eea6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:24.405712 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be093e8-20d5-429f-aeb5-7457d41eea6c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:24.414633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 460/913] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 05:25:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:24.415784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be093e8-20d5-429f-aeb5-7457d41eea6c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:24.469276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 454/914] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:25:24 2022] POST /identity/v3/services => generated 241 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:24.470049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135b216-4e86-4105-981f-416ca043f184 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:24.525341 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:24.556327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be093e8-20d5-429f-aeb5-7457d41eea6c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:24.610471 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135b216-4e86-4105-981f-416ca043f184 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:24.616435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59935e-17eb-4694-aa70-4ff48461520d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:24.617388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59935e-17eb-4694-aa70-4ff48461520d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:24.618379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59935e-17eb-4694-aa70-4ff48461520d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:24.618659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59935e-17eb-4694-aa70-4ff48461520d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:24.619471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59935e-17eb-4694-aa70-4ff48461520d None admin] RBAC: Adding query filter params (interface=public, service_id=ddde00daa94c4676a441a780d08a49d4, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:24.619705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59935e-17eb-4694-aa70-4ff48461520d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:24.621125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59935e-17eb-4694-aa70-4ff48461520d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:24.634500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 461/915] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 05:25:24 2022] GET /identity/v3/endpoints?service_id=ddde00daa94c4676a441a780d08a49d4&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:24.635968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59935e-17eb-4694-aa70-4ff48461520d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:24.651366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:24.663274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abced7-0f55-45a2-bd0d-31fc3f376ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:24.663909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abced7-0f55-45a2-bd0d-31fc3f376ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:24.666179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abced7-0f55-45a2-bd0d-31fc3f376ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:24.666179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abced7-0f55-45a2-bd0d-31fc3f376ae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:24.676409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abced7-0f55-45a2-bd0d-31fc3f376ae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:24.677906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abced7-0f55-45a2-bd0d-31fc3f376ae8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-34abced7-0f55-45a2-bd0d-31fc3f376ae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:24.694521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:24.695522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:24.696786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:24.696786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:24.696786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:24.696786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 455/916] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:24.701098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abced7-0f55-45a2-bd0d-31fc3f376ae8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:24.753261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59935e-17eb-4694-aa70-4ff48461520d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:24.793776 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3072e-5b7c-4fc7-9da7-e9d1cacbc532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:24.794714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3072e-5b7c-4fc7-9da7-e9d1cacbc532 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:24.795140 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3072e-5b7c-4fc7-9da7-e9d1cacbc532 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:24.795561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3072e-5b7c-4fc7-9da7-e9d1cacbc532 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:24.796326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3072e-5b7c-4fc7-9da7-e9d1cacbc532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:24.797954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3072e-5b7c-4fc7-9da7-e9d1cacbc532 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:24.810167 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 462/917] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 05:25:24 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:25:24.814553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:24.829234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abced7-0f55-45a2-bd0d-31fc3f376ae8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:24.874096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd1961-a1d2-4000-ae1a-e9ca993b392c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:24.874714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd1961-a1d2-4000-ae1a-e9ca993b392c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:24.874990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd1961-a1d2-4000-ae1a-e9ca993b392c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:24.875259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd1961-a1d2-4000-ae1a-e9ca993b392c None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:24.875910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd1961-a1d2-4000-ae1a-e9ca993b392c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:24.876212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd1961-a1d2-4000-ae1a-e9ca993b392c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:24.877580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd1961-a1d2-4000-ae1a-e9ca993b392c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:24.897998 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 456/918] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:24.903924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3072e-5b7c-4fc7-9da7-e9d1cacbc532 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:24.995263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:25.020549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3072e-5b7c-4fc7-9da7-e9d1cacbc532 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:25.061230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbac71-73a3-4df7-acf3-4c8f01392dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:25.061873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbac71-73a3-4df7-acf3-4c8f01392dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:25.062713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbac71-73a3-4df7-acf3-4c8f01392dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:25.063045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbac71-73a3-4df7-acf3-4c8f01392dc2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:25.071334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbac71-73a3-4df7-acf3-4c8f01392dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:25.072762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbac71-73a3-4df7-acf3-4c8f01392dc2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c7bbac71-73a3-4df7-acf3-4c8f01392dc2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:25.084754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:25.085718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:25.087129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 463/919] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 05:25:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:25.091556 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd1961-a1d2-4000-ae1a-e9ca993b392c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:25.196826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd1961-a1d2-4000-ae1a-e9ca993b392c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:25.233154 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd129a7-d226-499e-b0ac-e15dbfac02e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:25.233777 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd129a7-d226-499e-b0ac-e15dbfac02e6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:25.235001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd129a7-d226-499e-b0ac-e15dbfac02e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:25.235286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd129a7-d226-499e-b0ac-e15dbfac02e6 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:25.236024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd129a7-d226-499e-b0ac-e15dbfac02e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:25.236341 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd129a7-d226-499e-b0ac-e15dbfac02e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:25.237767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd129a7-d226-499e-b0ac-e15dbfac02e6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:25.256812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 457/920] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 05:25:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:25.262463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbac71-73a3-4df7-acf3-4c8f01392dc2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:25.372398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbac71-73a3-4df7-acf3-4c8f01392dc2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:25.407646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a4902-ac47-4001-89dc-b508a6de6184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:25.408287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a4902-ac47-4001-89dc-b508a6de6184 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:25.408565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a4902-ac47-4001-89dc-b508a6de6184 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:25.408841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a4902-ac47-4001-89dc-b508a6de6184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:25.410798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:25.420505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a4902-ac47-4001-89dc-b508a6de6184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:25.422752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a4902-ac47-4001-89dc-b508a6de6184 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-777a4902-ac47-4001-89dc-b508a6de6184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:25.438287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:25.439271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:25.440568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:25.440568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:25.440568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:25.440568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 464/921] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:25.445626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd129a7-d226-499e-b0ac-e15dbfac02e6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:25.449622 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:25.554227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd129a7-d226-499e-b0ac-e15dbfac02e6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:25.590000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e514f0d-d1dd-44c3-b7d3-7f1dae598504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:25.591017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e514f0d-d1dd-44c3-b7d3-7f1dae598504 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:25.591524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e514f0d-d1dd-44c3-b7d3-7f1dae598504 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:25.592011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e514f0d-d1dd-44c3-b7d3-7f1dae598504 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:25.592854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e514f0d-d1dd-44c3-b7d3-7f1dae598504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:25.593413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e514f0d-d1dd-44c3-b7d3-7f1dae598504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:25.595064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e514f0d-d1dd-44c3-b7d3-7f1dae598504 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:25.610184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 458/922] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:25.613302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a4902-ac47-4001-89dc-b508a6de6184 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:25.724400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a4902-ac47-4001-89dc-b508a6de6184 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:25.767465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141b65f-ca96-406c-84ba-269f9eea5ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:25.768123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141b65f-ca96-406c-84ba-269f9eea5ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:25.768386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141b65f-ca96-406c-84ba-269f9eea5ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:25.768662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141b65f-ca96-406c-84ba-269f9eea5ce0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:25.776117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141b65f-ca96-406c-84ba-269f9eea5ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:25.777616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141b65f-ca96-406c-84ba-269f9eea5ce0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-5141b65f-ca96-406c-84ba-269f9eea5ce0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:25.789682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:25.790613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:25.791523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:25.791523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:25.791523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:25.791523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:25.791523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:25.791523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:25.791523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:25.791523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:25.791523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 465/923] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:25.796500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e514f0d-d1dd-44c3-b7d3-7f1dae598504 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:25.863666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:25.886901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa143-b7b9-4e92-a302-7f162f2656f9 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:25.887184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa143-b7b9-4e92-a302-7f162f2656f9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:25.887465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa143-b7b9-4e92-a302-7f162f2656f9 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:25.889268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 466/924] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:25.895037 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89530292-1694-485f-9fa5-3114031fec2b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:25.895322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89530292-1694-485f-9fa5-3114031fec2b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:25.895735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89530292-1694-485f-9fa5-3114031fec2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:25.913250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e514f0d-d1dd-44c3-b7d3-7f1dae598504 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:25.920190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-89530292-1694-485f-9fa5-3114031fec2b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:25.921441 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-89530292-1694-485f-9fa5-3114031fec2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:25.951416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00605bd5-a236-4127-b9fc-b2c626b36298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:25.952420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00605bd5-a236-4127-b9fc-b2c626b36298 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:25.952929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00605bd5-a236-4127-b9fc-b2c626b36298 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:25.953384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00605bd5-a236-4127-b9fc-b2c626b36298 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:25.954445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00605bd5-a236-4127-b9fc-b2c626b36298 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:25.954935 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00605bd5-a236-4127-b9fc-b2c626b36298 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:25.957080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00605bd5-a236-4127-b9fc-b2c626b36298 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:25.969107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:25.970979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 459/925] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:25.980531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00605bd5-a236-4127-b9fc-b2c626b36298 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:26.006343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 467/926] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:26.081556 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00605bd5-a236-4127-b9fc-b2c626b36298 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:26.105335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:26.116682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da644f1-0730-44f6-be87-6dbca7815093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:26.117579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da644f1-0730-44f6-be87-6dbca7815093 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:26.117745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da644f1-0730-44f6-be87-6dbca7815093 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:26.118009 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da644f1-0730-44f6-be87-6dbca7815093 None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/5cf157961e3d45d5ad861f9b4f51ab50/roles/ad98bf0d9c9240e48f0243790aff940d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:26.120211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da644f1-0730-44f6-be87-6dbca7815093 None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=5cf157961e3d45d5ad861f9b4f51ab50, role_id=ad98bf0d9c9240e48f0243790aff940d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:26.121635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da644f1-0730-44f6-be87-6dbca7815093 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:26.223861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 460/927] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:25:25 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/5cf157961e3d45d5ad861f9b4f51ab50/roles/ad98bf0d9c9240e48f0243790aff940d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:25:26.282674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89530292-1694-485f-9fa5-3114031fec2b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:26.393734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89530292-1694-485f-9fa5-3114031fec2b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:26.431512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09030a89-1185-427c-a271-eca628a69394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:26.432619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09030a89-1185-427c-a271-eca628a69394 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:26.433130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09030a89-1185-427c-a271-eca628a69394 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:26.433602 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09030a89-1185-427c-a271-eca628a69394 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:26.434415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09030a89-1185-427c-a271-eca628a69394 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:26.435998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09030a89-1185-427c-a271-eca628a69394 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-09030a89-1185-427c-a271-eca628a69394 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:26.445605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:26.446645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 05:25:26.447494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:26.454219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 468/928] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:25:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:26.457115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da644f1-0730-44f6-be87-6dbca7815093 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:26.562702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da644f1-0730-44f6-be87-6dbca7815093 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:26.597778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbceb1d-4908-43e5-bfc0-cacf9632d213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:26.598783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbceb1d-4908-43e5-bfc0-cacf9632d213 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:26.599288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbceb1d-4908-43e5-bfc0-cacf9632d213 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:26.599775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbceb1d-4908-43e5-bfc0-cacf9632d213 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:26.600651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbceb1d-4908-43e5-bfc0-cacf9632d213 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:26.601151 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbceb1d-4908-43e5-bfc0-cacf9632d213 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:26.602812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbceb1d-4908-43e5-bfc0-cacf9632d213 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:26.611536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 461/929] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:25:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:26.614345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09030a89-1185-427c-a271-eca628a69394 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:26.658707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:26.723507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09030a89-1185-427c-a271-eca628a69394 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:26.758838 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7206249-ec76-4eba-8584-cc7aa1b58ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:26.759444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7206249-ec76-4eba-8584-cc7aa1b58ddc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:26.759721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7206249-ec76-4eba-8584-cc7aa1b58ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:26.759989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7206249-ec76-4eba-8584-cc7aa1b58ddc None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:26.760619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7206249-ec76-4eba-8584-cc7aa1b58ddc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:26.760919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7206249-ec76-4eba-8584-cc7aa1b58ddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:26.762304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7206249-ec76-4eba-8584-cc7aa1b58ddc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:26.768229 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 469/930] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 05:25:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:26.773695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbceb1d-4908-43e5-bfc0-cacf9632d213 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:26.832404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:26.890713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbceb1d-4908-43e5-bfc0-cacf9632d213 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:26.928937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701f77d-e0ea-4942-8f56-d51785bb285e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:26.929572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701f77d-e0ea-4942-8f56-d51785bb285e None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:26.929904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701f77d-e0ea-4942-8f56-d51785bb285e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:26.930115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701f77d-e0ea-4942-8f56-d51785bb285e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:26.930839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701f77d-e0ea-4942-8f56-d51785bb285e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:26.932305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701f77d-e0ea-4942-8f56-d51785bb285e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:26.942205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d0bba-8169-499d-baac-6b90eeed61d5 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:26.942534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d0bba-8169-499d-baac-6b90eeed61d5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:26.942768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d0bba-8169-499d-baac-6b90eeed61d5 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:26.945928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 470/931] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:26.950550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942151-575c-441d-90af-7352f09c5e5e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:26.951685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942151-575c-441d-90af-7352f09c5e5e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:26.951685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942151-575c-441d-90af-7352f09c5e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:26.969233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-80942151-575c-441d-90af-7352f09c5e5e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:26.970656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-80942151-575c-441d-90af-7352f09c5e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:27.010617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 462/932] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:25:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:27.036006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35fd9c-9036-482e-be11-fc4677dedae8 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:27.036374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35fd9c-9036-482e-be11-fc4677dedae8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:27.036560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35fd9c-9036-482e-be11-fc4677dedae8 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:27.038262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 463/933] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:27.043981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb361e12-048a-4820-9be1-7524b5bba815 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:27.044289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb361e12-048a-4820-9be1-7524b5bba815 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:27.044528 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb361e12-048a-4820-9be1-7524b5bba815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:27.060676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 471/934] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:27.065114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fb361e12-048a-4820-9be1-7524b5bba815 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:27.066362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fb361e12-048a-4820-9be1-7524b5bba815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:27.143302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 464/935] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:27.338316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80942151-575c-441d-90af-7352f09c5e5e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:27.368334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:27.412202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb361e12-048a-4820-9be1-7524b5bba815 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:27.443266 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80942151-575c-441d-90af-7352f09c5e5e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:27.478483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744ee84-e615-4b60-8857-ede030bc20dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:27.478729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:27.479140 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744ee84-e615-4b60-8857-ede030bc20dc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:27.479398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744ee84-e615-4b60-8857-ede030bc20dc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:27.479675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744ee84-e615-4b60-8857-ede030bc20dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:27.480261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744ee84-e615-4b60-8857-ede030bc20dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:27.481630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744ee84-e615-4b60-8857-ede030bc20dc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-1744ee84-e615-4b60-8857-ede030bc20dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:27.491439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:27.492485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:27.493322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 472/936] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:27.499074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744ee84-e615-4b60-8857-ede030bc20dc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:27.520933 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb361e12-048a-4820-9be1-7524b5bba815 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:27.558203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f083a50-5b4f-43de-a186-568297fcf787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:27.559245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083a50-5b4f-43de-a186-568297fcf787 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:27.559739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083a50-5b4f-43de-a186-568297fcf787 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:27.560225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083a50-5b4f-43de-a186-568297fcf787 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:27.561008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f083a50-5b4f-43de-a186-568297fcf787 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:27.562624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f083a50-5b4f-43de-a186-568297fcf787 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5f083a50-5b4f-43de-a186-568297fcf787 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:27.577973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:27.579050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 05:25:27.579952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:27.587965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 465/937] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Oct 4 05:25:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:27.588769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f083a50-5b4f-43de-a186-568297fcf787 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:27.638896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744ee84-e615-4b60-8857-ede030bc20dc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:27.642453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:27.704341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ad326-a0f8-4989-839a-00e901d26ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:27.705381 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ad326-a0f8-4989-839a-00e901d26ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:27.705872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ad326-a0f8-4989-839a-00e901d26ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:27.706368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ad326-a0f8-4989-839a-00e901d26ef3 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:27.707198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ad326-a0f8-4989-839a-00e901d26ef3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:27.708085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ad326-a0f8-4989-839a-00e901d26ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:27.711224 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ad326-a0f8-4989-839a-00e901d26ef3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:27.721737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 473/938] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 05:25:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:27.731141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ad326-a0f8-4989-839a-00e901d26ef3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:27.745666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:27.758648 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f083a50-5b4f-43de-a186-568297fcf787 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:27.818261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468284f2-c380-454d-81d3-25c0fc121159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:27.819478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468284f2-c380-454d-81d3-25c0fc121159 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:27.819957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468284f2-c380-454d-81d3-25c0fc121159 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:27.820435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468284f2-c380-454d-81d3-25c0fc121159 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:27.821237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468284f2-c380-454d-81d3-25c0fc121159 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:27.821707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468284f2-c380-454d-81d3-25c0fc121159 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:27.823298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468284f2-c380-454d-81d3-25c0fc121159 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:27.834998 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 466/939] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:27.837014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468284f2-c380-454d-81d3-25c0fc121159 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:27.944920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ad326-a0f8-4989-839a-00e901d26ef3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:27.966125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468284f2-c380-454d-81d3-25c0fc121159 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:28.028127 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab44f6d-4df7-4255-8f72-2fb546616f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:28.030094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab44f6d-4df7-4255-8f72-2fb546616f25 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:28.031066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab44f6d-4df7-4255-8f72-2fb546616f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:28.031854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab44f6d-4df7-4255-8f72-2fb546616f25 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:28.033300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab44f6d-4df7-4255-8f72-2fb546616f25 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:28.034193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc0d2d-d7b3-4809-8362-b4230dabf972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:28.034753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab44f6d-4df7-4255-8f72-2fb546616f25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:28.035555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc0d2d-d7b3-4809-8362-b4230dabf972 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:28.037135 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab44f6d-4df7-4255-8f72-2fb546616f25 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:28.037638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc0d2d-d7b3-4809-8362-b4230dabf972 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:28.038208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc0d2d-d7b3-4809-8362-b4230dabf972 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:28.039049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc0d2d-d7b3-4809-8362-b4230dabf972 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:28.040966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc0d2d-d7b3-4809-8362-b4230dabf972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:28.044020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc0d2d-d7b3-4809-8362-b4230dabf972 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:28.055288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 467/940] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:28.056786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab44f6d-4df7-4255-8f72-2fb546616f25 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:28.061473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 474/941] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 05:25:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:28.070089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc0d2d-d7b3-4809-8362-b4230dabf972 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:28.104961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:28.109409 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:28.186474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab44f6d-4df7-4255-8f72-2fb546616f25 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:28.191704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc0d2d-d7b3-4809-8362-b4230dabf972 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:28.226076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee109a3-b835-48be-8d78-875febf33292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:28.226794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee109a3-b835-48be-8d78-875febf33292 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:28.227168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee109a3-b835-48be-8d78-875febf33292 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:28.227531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee109a3-b835-48be-8d78-875febf33292 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:28.228286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee109a3-b835-48be-8d78-875febf33292 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:28.229931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee109a3-b835-48be-8d78-875febf33292 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:28.230386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7b65f-f7c7-43b9-873e-e478723ef7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:28.231156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7b65f-f7c7-43b9-873e-e478723ef7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:28.231596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7b65f-f7c7-43b9-873e-e478723ef7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:28.232012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7b65f-f7c7-43b9-873e-e478723ef7a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:28.232852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7b65f-f7c7-43b9-873e-e478723ef7a0 None admin] RBAC: Adding query filter params (interface=public, service_id=13770568bf9e4296b34a1c10f3164d46, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:28.233303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7b65f-f7c7-43b9-873e-e478723ef7a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:28.234893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7b65f-f7c7-43b9-873e-e478723ef7a0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:28.241075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 475/942] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 05:25:28 2022] GET /identity/v3/endpoints?service_id=13770568bf9e4296b34a1c10f3164d46&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:28.247022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7b65f-f7c7-43b9-873e-e478723ef7a0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:28.312842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 468/943] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:25:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:28.351687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:28.369868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7b65f-f7c7-43b9-873e-e478723ef7a0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:28.419656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2617c4-10c1-4bfc-8334-7a9f15f5ece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:28.420632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2617c4-10c1-4bfc-8334-7a9f15f5ece6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:28.421096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2617c4-10c1-4bfc-8334-7a9f15f5ece6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:28.422064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2617c4-10c1-4bfc-8334-7a9f15f5ece6 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:28.422874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2617c4-10c1-4bfc-8334-7a9f15f5ece6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:28.424430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2617c4-10c1-4bfc-8334-7a9f15f5ece6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:28.436149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 476/944] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 05:25:28 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:25:28.507950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ba8d0-dc5b-4776-826e-de72a7843d67 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:28.508442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ba8d0-dc5b-4776-826e-de72a7843d67 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:28.508442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ba8d0-dc5b-4776-826e-de72a7843d67 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:28.510104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 469/945] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:28.515932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e944-e5e7-4895-9293-62be9c4ec0d9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:28.516246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e944-e5e7-4895-9293-62be9c4ec0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:28.516546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e944-e5e7-4895-9293-62be9c4ec0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:28.535900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1e92e944-e5e7-4895-9293-62be9c4ec0d9 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:28.537118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1e92e944-e5e7-4895-9293-62be9c4ec0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:28.612914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 477/946] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:28.896440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ba8d0-dc5b-4776-826e-de72a7843d67 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:28.980234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:28.999257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ba8d0-dc5b-4776-826e-de72a7843d67 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:29.034453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0068033-855f-4dae-87ae-5411eb3ccb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:29.035085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0068033-855f-4dae-87ae-5411eb3ccb80 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:29.035360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0068033-855f-4dae-87ae-5411eb3ccb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:29.035631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0068033-855f-4dae-87ae-5411eb3ccb80 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:29.045880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0068033-855f-4dae-87ae-5411eb3ccb80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c0068033-855f-4dae-87ae-5411eb3ccb80 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:29.051340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:29.052286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:29.053146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 470/947] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 05:25:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:29.058540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e944-e5e7-4895-9293-62be9c4ec0d9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:29.140924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:29.164294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e944-e5e7-4895-9293-62be9c4ec0d9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:29.203895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f24f1a-a05e-45fb-b226-87b21450aeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:29.204522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f24f1a-a05e-45fb-b226-87b21450aeda None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:29.204797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f24f1a-a05e-45fb-b226-87b21450aeda None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:29.205067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f24f1a-a05e-45fb-b226-87b21450aeda None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:29.205743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f24f1a-a05e-45fb-b226-87b21450aeda None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:29.206040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f24f1a-a05e-45fb-b226-87b21450aeda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:29.207543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f24f1a-a05e-45fb-b226-87b21450aeda None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:29.216805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 478/948] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 05:25:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:29.222717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0068033-855f-4dae-87ae-5411eb3ccb80 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:29.301791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d012b-1fa9-42e6-a275-5333632bfd29 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:29.302127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d012b-1fa9-42e6-a275-5333632bfd29 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:29.302386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d012b-1fa9-42e6-a275-5333632bfd29 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:29.304102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 479/949] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:29.309673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161ffc6-7356-49a6-be15-80020beba69d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:29.309963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161ffc6-7356-49a6-be15-80020beba69d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:29.310250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161ffc6-7356-49a6-be15-80020beba69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:29.329393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5161ffc6-7356-49a6-be15-80020beba69d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:29.330756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5161ffc6-7356-49a6-be15-80020beba69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:29.333382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0068033-855f-4dae-87ae-5411eb3ccb80 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:29.372252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358da79-1d76-4d2a-a720-7c083527e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:29.373315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358da79-1d76-4d2a-a720-7c083527e191 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:29.373778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358da79-1d76-4d2a-a720-7c083527e191 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:29.374269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358da79-1d76-4d2a-a720-7c083527e191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:29.385740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358da79-1d76-4d2a-a720-7c083527e191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:29.387439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358da79-1d76-4d2a-a720-7c083527e191 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:29.389458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1358da79-1d76-4d2a-a720-7c083527e191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:29.406429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:29.407383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:29.408280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:29.408280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:29.408280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:29.414208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 471/951] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:29.414385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 480/951] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:29.416082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161ffc6-7356-49a6-be15-80020beba69d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:29.487608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:29.523533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161ffc6-7356-49a6-be15-80020beba69d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:29.559604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577edc4e-e561-4697-8e5a-9f108bac92fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:29.560218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577edc4e-e561-4697-8e5a-9f108bac92fb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:29.560497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577edc4e-e561-4697-8e5a-9f108bac92fb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:29.560768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577edc4e-e561-4697-8e5a-9f108bac92fb None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:29.561408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577edc4e-e561-4697-8e5a-9f108bac92fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:29.561711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577edc4e-e561-4697-8e5a-9f108bac92fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:29.563623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577edc4e-e561-4697-8e5a-9f108bac92fb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:29.575925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 481/952] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:29.581968 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358da79-1d76-4d2a-a720-7c083527e191 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:29.683364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358da79-1d76-4d2a-a720-7c083527e191 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:29.689881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577edc4e-e561-4697-8e5a-9f108bac92fb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:29.719434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd9168-4dc4-4e17-ae6d-d33ea0749328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:29.720067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd9168-4dc4-4e17-ae6d-d33ea0749328 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:29.720342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd9168-4dc4-4e17-ae6d-d33ea0749328 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:29.720613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd9168-4dc4-4e17-ae6d-d33ea0749328 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:29.728633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd9168-4dc4-4e17-ae6d-d33ea0749328 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:29.730066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd9168-4dc4-4e17-ae6d-d33ea0749328 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-eefd9168-4dc4-4e17-ae6d-d33ea0749328 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:29.741394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:29.742408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:29.743380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 472/953] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 05:25:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:29.748966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd9168-4dc4-4e17-ae6d-d33ea0749328 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:29.815588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577edc4e-e561-4697-8e5a-9f108bac92fb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:29.817985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:29.850301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:29.860716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:29.861334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:29.861616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:29.861891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:29.862575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:29.864266 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:29.868103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WARNING keystone.server.flask.common [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 05:25:29.872373 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd9168-4dc4-4e17-ae6d-d33ea0749328 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:29.912931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ce1df-1ca1-46ec-bdce-06232767757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:29.913541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ce1df-1ca1-46ec-bdce-06232767757e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:29.913816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ce1df-1ca1-46ec-bdce-06232767757e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:29.914092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ce1df-1ca1-46ec-bdce-06232767757e None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:29.914847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ce1df-1ca1-46ec-bdce-06232767757e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:29.915157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ce1df-1ca1-46ec-bdce-06232767757e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:29.916692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ce1df-1ca1-46ec-bdce-06232767757e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:29.930474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.sql.core [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 05:25:29.930474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 05:25:29.930474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [parameters: {'id': '976ab82a265b4545942526a6357200b5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 05:25:29.930474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81481) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 05:25:29.940316 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 473/954] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 05:25:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:29.947199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ce1df-1ca1-46ec-bdce-06232767757e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 976ab82a265b4545942526a6357200b5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 976ab82a265b4545942526a6357200b5. Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application conn.query(q) Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 05:25:29.947837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result.read() Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 04 05:25:29.948664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application next(self.gen) Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application yield resource Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application next(self.gen) Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise self.value Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application session.commit() Oct 04 05:25:29.949736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.session.flush() Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application compat.raise_( Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 05:25:29.950687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 05:25:29.951697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application conn.query(q) Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application result.read() Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 05:25:29.952570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [parameters: {'id': '976ab82a265b4545942526a6357200b5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:29.953450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:29.955599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 976ab82a265b4545942526a6357200b5. Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 482/955] 10.222.0.26 () {60 vars in 1140 bytes} [Tue Oct 4 05:25:29 2022] POST /identity/v3/projects => generated 262 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:29.957842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:30.067898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258e444-ceee-46b6-a43e-abf0092646db None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:30.100649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ce1df-1ca1-46ec-bdce-06232767757e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:30.103021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af18766-bd76-4b78-bcdf-3e2cf06c2843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:30.103659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af18766-bd76-4b78-bcdf-3e2cf06c2843 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:30.103946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af18766-bd76-4b78-bcdf-3e2cf06c2843 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:30.104246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af18766-bd76-4b78-bcdf-3e2cf06c2843 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:30.112349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af18766-bd76-4b78-bcdf-3e2cf06c2843 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:30.114201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af18766-bd76-4b78-bcdf-3e2cf06c2843 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:30.119168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7af18766-bd76-4b78-bcdf-3e2cf06c2843 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:30.126902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:30.130274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:30.131212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:30.131212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:30.131212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:30.131212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:30.131212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:30.131212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:30.131212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:30.131212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:30.132914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 483/956] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:30.133808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af18766-bd76-4b78-bcdf-3e2cf06c2843 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:30.153634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2f291-600a-4e87-a034-9e8a65c62e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:30.155055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2f291-600a-4e87-a034-9e8a65c62e45 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:30.155563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2f291-600a-4e87-a034-9e8a65c62e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:30.156036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2f291-600a-4e87-a034-9e8a65c62e45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:30.183969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:30.244934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2f291-600a-4e87-a034-9e8a65c62e45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:30.246355 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2f291-600a-4e87-a034-9e8a65c62e45 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:30.254687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af18766-bd76-4b78-bcdf-3e2cf06c2843 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:30.293937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abf148-9372-42de-8bfb-cf88f1175c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:30.294403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf148-9372-42de-8bfb-cf88f1175c5b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:30.294690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf148-9372-42de-8bfb-cf88f1175c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:30.294965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf148-9372-42de-8bfb-cf88f1175c5b None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:30.295643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abf148-9372-42de-8bfb-cf88f1175c5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:30.295951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abf148-9372-42de-8bfb-cf88f1175c5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:30.298008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abf148-9372-42de-8bfb-cf88f1175c5b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:30.326588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 484/957] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:30.372221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 474/958] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:29 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 05:25:30.378784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abf148-9372-42de-8bfb-cf88f1175c5b None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:30.497011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abf148-9372-42de-8bfb-cf88f1175c5b None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:30.500510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:30.539241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a197eb1-bcd7-4a98-9609-36725849cddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:30.540303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197eb1-bcd7-4a98-9609-36725849cddc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:30.540863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197eb1-bcd7-4a98-9609-36725849cddc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:30.541345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a197eb1-bcd7-4a98-9609-36725849cddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:30.555920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a197eb1-bcd7-4a98-9609-36725849cddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:30.558315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a197eb1-bcd7-4a98-9609-36725849cddc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8a197eb1-bcd7-4a98-9609-36725849cddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:30.576024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:30.577045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:30.578107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:30.578107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:30.578107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:30.578911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 485/959] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:30.584761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2f291-600a-4e87-a034-9e8a65c62e45 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:30.600744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:30.625101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f56e51-af19-4114-bb03-322ea8a255f7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:30.625504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f56e51-af19-4114-bb03-322ea8a255f7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:30.625780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f56e51-af19-4114-bb03-322ea8a255f7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:30.627478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 486/960] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:30.632701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173edbc-611e-4779-a61e-c8aca55fac70 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:30.632981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173edbc-611e-4779-a61e-c8aca55fac70 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:30.633253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173edbc-611e-4779-a61e-c8aca55fac70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:30.652663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5173edbc-611e-4779-a61e-c8aca55fac70 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:30.654940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5173edbc-611e-4779-a61e-c8aca55fac70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:30.695089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2f291-600a-4e87-a034-9e8a65c62e45 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:30.729170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 487/961] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:30.730599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c379e-442e-407f-b14f-64c3b7837386 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:30.730938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c379e-442e-407f-b14f-64c3b7837386 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:30.731285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c379e-442e-407f-b14f-64c3b7837386 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:30.734269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 488/962] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:30.738670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51fd60-72b7-45b4-9017-086892b7f2e9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:30.738958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51fd60-72b7-45b4-9017-086892b7f2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:30.739238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51fd60-72b7-45b4-9017-086892b7f2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:30.743384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfba80-950e-415a-bafd-e1248342c82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:30.744009 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfba80-950e-415a-bafd-e1248342c82f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:30.744279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfba80-950e-415a-bafd-e1248342c82f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:30.744554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfba80-950e-415a-bafd-e1248342c82f None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:30.745200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfba80-950e-415a-bafd-e1248342c82f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:30.745505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfba80-950e-415a-bafd-e1248342c82f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:30.746878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfba80-950e-415a-bafd-e1248342c82f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:30.759142 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 475/963] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:30.762369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9d51fd60-72b7-45b4-9017-086892b7f2e9 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:30.763694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9d51fd60-72b7-45b4-9017-086892b7f2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:30.764184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfba80-950e-415a-bafd-e1248342c82f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:30.851777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 489/964] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:30.876719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:30.897699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfba80-950e-415a-bafd-e1248342c82f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:30.933557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323faec-8254-452d-a87e-77ea174301b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:30.934593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323faec-8254-452d-a87e-77ea174301b6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:30.935250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323faec-8254-452d-a87e-77ea174301b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:30.935527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323faec-8254-452d-a87e-77ea174301b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:30.943071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323faec-8254-452d-a87e-77ea174301b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:30.944868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323faec-8254-452d-a87e-77ea174301b6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-9323faec-8254-452d-a87e-77ea174301b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:30.956771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:30.957666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:30.958616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:30.958616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:30.958616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:30.958616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:30.958616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:30.958616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:30.958616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:30.958616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:30.958616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 476/965] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:30.964023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51fd60-72b7-45b4-9017-086892b7f2e9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:30.983339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:31.001252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323faec-8254-452d-a87e-77ea174301b6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:31.068368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51fd60-72b7-45b4-9017-086892b7f2e9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:31.103115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fa62e-7b9c-4966-8dd9-341e65608339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:31.103845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa62e-7b9c-4966-8dd9-341e65608339 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:31.104124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa62e-7b9c-4966-8dd9-341e65608339 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:31.104395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fa62e-7b9c-4966-8dd9-341e65608339 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:31.105081 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fa62e-7b9c-4966-8dd9-341e65608339 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:31.105381 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fa62e-7b9c-4966-8dd9-341e65608339 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:31.107044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fa62e-7b9c-4966-8dd9-341e65608339 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:31.119939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 490/966] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:31.126509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fa62e-7b9c-4966-8dd9-341e65608339 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:31.133757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323faec-8254-452d-a87e-77ea174301b6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:31.165216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:31.170378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d813f8-92ad-4643-a4de-0ea7ed2065d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:31.176705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d813f8-92ad-4643-a4de-0ea7ed2065d2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:31.176705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d813f8-92ad-4643-a4de-0ea7ed2065d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:31.176705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d813f8-92ad-4643-a4de-0ea7ed2065d2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:31.176705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d813f8-92ad-4643-a4de-0ea7ed2065d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:31.176705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d813f8-92ad-4643-a4de-0ea7ed2065d2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e8d813f8-92ad-4643-a4de-0ea7ed2065d2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:31.185455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:31.186567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:31.187469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 477/967] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:31.188379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d813f8-92ad-4643-a4de-0ea7ed2065d2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:31.213856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:31.237569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fa62e-7b9c-4966-8dd9-341e65608339 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:31.274831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b7bec-c240-445e-b4ac-3c1b437c48bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:31.275475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b7bec-c240-445e-b4ac-3c1b437c48bc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:31.275732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b7bec-c240-445e-b4ac-3c1b437c48bc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:31.275991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b7bec-c240-445e-b4ac-3c1b437c48bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:31.276817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b7bec-c240-445e-b4ac-3c1b437c48bc None admin] RBAC: Adding query filter params (role.id=ad98bf0d9c9240e48f0243790aff940d, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=5cf157961e3d45d5ad861f9b4f51ab50) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:31.277021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b7bec-c240-445e-b4ac-3c1b437c48bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:31.278489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b7bec-c240-445e-b4ac-3c1b437c48bc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:31.294169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 491/968] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:31 2022] GET /identity/v3/role_assignments?user.id=5cf157961e3d45d5ad861f9b4f51ab50&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=ad98bf0d9c9240e48f0243790aff940d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:31.294169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b7bec-c240-445e-b4ac-3c1b437c48bc None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:31.310340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d813f8-92ad-4643-a4de-0ea7ed2065d2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:31.353709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7143c84-b1b7-4320-adae-3382d2ec30b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:31.354403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7143c84-b1b7-4320-adae-3382d2ec30b2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:31.354688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7143c84-b1b7-4320-adae-3382d2ec30b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:31.354961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7143c84-b1b7-4320-adae-3382d2ec30b2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:31.355558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7143c84-b1b7-4320-adae-3382d2ec30b2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:31.356939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7143c84-b1b7-4320-adae-3382d2ec30b2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e7143c84-b1b7-4320-adae-3382d2ec30b2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:31.368470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:31.369580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:31.370491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:31.374631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 478/969] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:31.376181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7143c84-b1b7-4320-adae-3382d2ec30b2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:31.422115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b7bec-c240-445e-b4ac-3c1b437c48bc None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:31.451866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:31.474319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf53f0e-78fb-4542-b3a9-001c8e01f449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:31.475721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf53f0e-78fb-4542-b3a9-001c8e01f449 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:31.476314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf53f0e-78fb-4542-b3a9-001c8e01f449 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:31.476768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf53f0e-78fb-4542-b3a9-001c8e01f449 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:31.477616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf53f0e-78fb-4542-b3a9-001c8e01f449 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:31.478320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf53f0e-78fb-4542-b3a9-001c8e01f449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:31.480625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf53f0e-78fb-4542-b3a9-001c8e01f449 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:31.488458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 492/970] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:31.494248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf53f0e-78fb-4542-b3a9-001c8e01f449 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:31.499192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:31.504467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7143c84-b1b7-4320-adae-3382d2ec30b2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:31.551591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5383d6-e3af-48b7-9364-0ca72955efe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:31.552660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5383d6-e3af-48b7-9364-0ca72955efe6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:31.552792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5383d6-e3af-48b7-9364-0ca72955efe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:31.553038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5383d6-e3af-48b7-9364-0ca72955efe6 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:31.553668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5383d6-e3af-48b7-9364-0ca72955efe6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:31.554159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5383d6-e3af-48b7-9364-0ca72955efe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:31.555955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5383d6-e3af-48b7-9364-0ca72955efe6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:31.561481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 479/971] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 05:25:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:31.566909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5383d6-e3af-48b7-9364-0ca72955efe6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:31.621588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf53f0e-78fb-4542-b3a9-001c8e01f449 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:31.658629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a6146-5521-402b-ab49-736ab9ac0085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:31.659725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6146-5521-402b-ab49-736ab9ac0085 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:31.660304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6146-5521-402b-ab49-736ab9ac0085 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:31.660860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6146-5521-402b-ab49-736ab9ac0085 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:31.661599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a6146-5521-402b-ab49-736ab9ac0085 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:31.662099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a6146-5521-402b-ab49-736ab9ac0085 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:31.663833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a6146-5521-402b-ab49-736ab9ac0085 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:31.669106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5383d6-e3af-48b7-9364-0ca72955efe6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:31.669653 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 493/972] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:31.701586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:31.714539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ca056-a041-471e-91fc-c2f9aef02ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:31.715403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ca056-a041-471e-91fc-c2f9aef02ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:31.716899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ca056-a041-471e-91fc-c2f9aef02ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:31.716899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ca056-a041-471e-91fc-c2f9aef02ba1 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:31.716899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ca056-a041-471e-91fc-c2f9aef02ba1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:31.717369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ca056-a041-471e-91fc-c2f9aef02ba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:31.719607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ca056-a041-471e-91fc-c2f9aef02ba1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:31.725900 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 480/973] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 05:25:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:31.735300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a6146-5521-402b-ab49-736ab9ac0085 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:31.807819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:31.845755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a6146-5521-402b-ab49-736ab9ac0085 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:31.881410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ac328-37ee-4f83-86c7-8126366f4c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:31.881952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ac328-37ee-4f83-86c7-8126366f4c10 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:31.882252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ac328-37ee-4f83-86c7-8126366f4c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:31.882527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ac328-37ee-4f83-86c7-8126366f4c10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:31.883202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ac328-37ee-4f83-86c7-8126366f4c10 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:31.884573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ac328-37ee-4f83-86c7-8126366f4c10 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:32.045930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 494/974] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:25:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:32.665653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5f93d-f64c-4427-83bb-54ea89dae623 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:32.665991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5f93d-f64c-4427-83bb-54ea89dae623 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:32.666246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5f93d-f64c-4427-83bb-54ea89dae623 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:32.667839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 481/975] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:32.673759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03f533-04cb-46f9-bdde-ea978fbd1dba None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:32.674234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03f533-04cb-46f9-bdde-ea978fbd1dba None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:32.674527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03f533-04cb-46f9-bdde-ea978fbd1dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:32.694414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2e03f533-04cb-46f9-bdde-ea978fbd1dba None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:32.695651 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2e03f533-04cb-46f9-bdde-ea978fbd1dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:32.770952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 495/976] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:33.043147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5f93d-f64c-4427-83bb-54ea89dae623 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:33.150157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5f93d-f64c-4427-83bb-54ea89dae623 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:33.184068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:33.184673 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:33.184947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:33.185219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:33.185863 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:33.187918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:33.190459 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: WARNING keystone.server.flask.common [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 05:25:33.233412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.sql.core [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 05:25:33.233412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 05:25:33.233412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [parameters: {'user_id': 'e6f8b468cf0448c3bccaaa376e60ea0a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 05:25:33.233412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81480) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Oct 04 05:25:33.240780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 05:25:33.245662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application yield resource Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application next(self.gen) Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise self.value Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application session.commit() Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 05:25:33.246667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.session.flush() Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application compat.raise_( Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 05:25:33.247989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 05:25:33.249095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application conn.query(q) Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application result.read() Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 05:25:33.250834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e6f8b468cf0448c3bccaaa376e60ea0a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:33.251877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 04 05:25:33.252850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 04 05:25:33.253979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 04 05:25:33.253979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 04 05:25:33.253979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 05:25:33.253979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 05:25:33.253979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 05:25:33.253979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:33.253979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 482/977] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Oct 4 05:25:33 2022] POST /identity/v3/users => generated 160 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:33.258537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03f533-04cb-46f9-bdde-ea978fbd1dba None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:33.310195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:33.365114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03f533-04cb-46f9-bdde-ea978fbd1dba None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:33.401283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471e09a-77af-4ea0-ad79-605def7f794b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:33.402310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471e09a-77af-4ea0-ad79-605def7f794b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:33.402800 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471e09a-77af-4ea0-ad79-605def7f794b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:33.403280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471e09a-77af-4ea0-ad79-605def7f794b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:33.411577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471e09a-77af-4ea0-ad79-605def7f794b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:33.413285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471e09a-77af-4ea0-ad79-605def7f794b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9471e09a-77af-4ea0-ad79-605def7f794b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:33.426123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:33.427219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:33.428178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:33.428178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:33.428178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:33.428178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:33.428178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:33.428178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:33.428178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:33.428178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:33.428178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 05:25:33.428178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:33.430693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 496/978] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:25:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:33.436934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:33.490100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:33.543943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ff911-461f-4e9c-af12-5fc2deb1f182 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:33.581394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fab09-6d66-46af-af1e-ccc5334de158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:33.581937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fab09-6d66-46af-af1e-ccc5334de158 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:33.582319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fab09-6d66-46af-af1e-ccc5334de158 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:33.582594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fab09-6d66-46af-af1e-ccc5334de158 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:33.583318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fab09-6d66-46af-af1e-ccc5334de158 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:33.583623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fab09-6d66-46af-af1e-ccc5334de158 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:33.585309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fab09-6d66-46af-af1e-ccc5334de158 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:33.625998 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 483/979] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 05:25:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:33.690898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed585d17-50af-4e27-be7d-cc060b3107b0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:33.691424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed585d17-50af-4e27-be7d-cc060b3107b0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:33.691424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed585d17-50af-4e27-be7d-cc060b3107b0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:33.693212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 497/980] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:33.699492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec673f-34b3-42d2-b511-7f2758af465f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:33.699833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec673f-34b3-42d2-b511-7f2758af465f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:33.700046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec673f-34b3-42d2-b511-7f2758af465f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:33.720877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-94ec673f-34b3-42d2-b511-7f2758af465f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:33.722481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-94ec673f-34b3-42d2-b511-7f2758af465f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:33.796877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 484/981] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:33.979029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ad7ab-ec51-45a0-b055-12f3261aace2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:33.979313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ad7ab-ec51-45a0-b055-12f3261aace2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:33.979581 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ad7ab-ec51-45a0-b055-12f3261aace2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:33.980950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 498/982] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:33.986123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dde91-414e-44cd-9c4f-b20aaad1a79b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:33.986445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dde91-414e-44cd-9c4f-b20aaad1a79b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:33.986725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20dde91-414e-44cd-9c4f-b20aaad1a79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:34.005344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b20dde91-414e-44cd-9c4f-b20aaad1a79b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:34.006573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b20dde91-414e-44cd-9c4f-b20aaad1a79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:34.037467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:34.067491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ad7ab-ec51-45a0-b055-12f3261aace2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:34.082193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 485/983] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:34.174042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ad7ab-ec51-45a0-b055-12f3261aace2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:34.186176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:34.210208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e953231-06a6-4711-84ef-38853ee5fcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:34.210841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e953231-06a6-4711-84ef-38853ee5fcee None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:34.211118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e953231-06a6-4711-84ef-38853ee5fcee None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:34.211390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e953231-06a6-4711-84ef-38853ee5fcee None admin] PATH_INFO: `/v3/services/image` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:34.211968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e953231-06a6-4711-84ef-38853ee5fcee None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:34.213331 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e953231-06a6-4711-84ef-38853ee5fcee None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-2e953231-06a6-4711-84ef-38853ee5fcee None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:34.223000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:34.223974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 05:25:34.224862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:34.226493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 499/984] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Oct 4 05:25:34 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:34.230041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dde91-414e-44cd-9c4f-b20aaad1a79b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:34.272859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70fd52-cdde-4595-8612-a88867d1d5ca None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:34.273177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70fd52-cdde-4595-8612-a88867d1d5ca None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:34.273405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70fd52-cdde-4595-8612-a88867d1d5ca None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:34.274767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 500/985] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:34.279510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93c905-3095-4916-8242-ae61731cb790 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:34.279790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93c905-3095-4916-8242-ae61731cb790 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:34.280059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93c905-3095-4916-8242-ae61731cb790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:34.297745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-aa93c905-3095-4916-8242-ae61731cb790 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:34.298946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aa93c905-3095-4916-8242-ae61731cb790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:34.341912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20dde91-414e-44cd-9c4f-b20aaad1a79b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:34.368599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 501/986] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:34.369407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93c905-3095-4916-8242-ae61731cb790 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:34.378866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bcf59-f81e-486f-84fb-174895e91c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:34.379471 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bcf59-f81e-486f-84fb-174895e91c73 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:34.379747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bcf59-f81e-486f-84fb-174895e91c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:34.380018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bcf59-f81e-486f-84fb-174895e91c73 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:34.380640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bcf59-f81e-486f-84fb-174895e91c73 None admin] RBAC: Adding query filter params (name=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:34.380941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bcf59-f81e-486f-84fb-174895e91c73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:34.382312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bcf59-f81e-486f-84fb-174895e91c73 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:34.387135 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 486/987] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:34.392570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bcf59-f81e-486f-84fb-174895e91c73 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:34.428819 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:34.476720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93c905-3095-4916-8242-ae61731cb790 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:34.479065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:34.501586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bcf59-f81e-486f-84fb-174895e91c73 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:34.512317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662c13b-685e-41f1-8d8d-4acd843ea0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:34.512928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662c13b-685e-41f1-8d8d-4acd843ea0f2 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:34.513364 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662c13b-685e-41f1-8d8d-4acd843ea0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:34.513482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662c13b-685e-41f1-8d8d-4acd843ea0f2 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:34.514087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662c13b-685e-41f1-8d8d-4acd843ea0f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:34.515513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662c13b-685e-41f1-8d8d-4acd843ea0f2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:34.536046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344595c-7c48-4b36-92a7-6415cc696717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:34.536644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344595c-7c48-4b36-92a7-6415cc696717 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:34.536920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344595c-7c48-4b36-92a7-6415cc696717 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:34.537191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344595c-7c48-4b36-92a7-6415cc696717 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:34.537869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344595c-7c48-4b36-92a7-6415cc696717 None admin] RBAC: Adding query filter params (type=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:34.538191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344595c-7c48-4b36-92a7-6415cc696717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:34.539523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344595c-7c48-4b36-92a7-6415cc696717 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:34.554165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 487/988] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:34.586645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 502/989] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:25:34 2022] POST /identity/v3/services => generated 242 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:34.640887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344595c-7c48-4b36-92a7-6415cc696717 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:34.754526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:34.761503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344595c-7c48-4b36-92a7-6415cc696717 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:34.806084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aafa35-8a1e-4928-9a77-e54ef9bfeb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:34.806748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aafa35-8a1e-4928-9a77-e54ef9bfeb50 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:34.807028 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aafa35-8a1e-4928-9a77-e54ef9bfeb50 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:34.807304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aafa35-8a1e-4928-9a77-e54ef9bfeb50 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:34.807887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aafa35-8a1e-4928-9a77-e54ef9bfeb50 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:34.809239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aafa35-8a1e-4928-9a77-e54ef9bfeb50 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-83aafa35-8a1e-4928-9a77-e54ef9bfeb50 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:34.819174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:34.820208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:34.821056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 488/990] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:34.828050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662c13b-685e-41f1-8d8d-4acd843ea0f2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:34.934105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662c13b-685e-41f1-8d8d-4acd843ea0f2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:34.969500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53a72b-b815-4108-9567-58f810e27989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:34.970121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53a72b-b815-4108-9567-58f810e27989 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:34.970428 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53a72b-b815-4108-9567-58f810e27989 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:34.970703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53a72b-b815-4108-9567-58f810e27989 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:34.971336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53a72b-b815-4108-9567-58f810e27989 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:34.971640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53a72b-b815-4108-9567-58f810e27989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:34.973047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53a72b-b815-4108-9567-58f810e27989 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:34.978105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 503/991] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 05:25:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:34.984500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aafa35-8a1e-4928-9a77-e54ef9bfeb50 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:35.084564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aafa35-8a1e-4928-9a77-e54ef9bfeb50 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:35.119740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab5179-5292-45ec-b0d1-d7f0ab8b30a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:35.120348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab5179-5292-45ec-b0d1-d7f0ab8b30a3 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:35.120629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab5179-5292-45ec-b0d1-d7f0ab8b30a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:35.120901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab5179-5292-45ec-b0d1-d7f0ab8b30a3 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:35.121525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab5179-5292-45ec-b0d1-d7f0ab8b30a3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:35.121828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab5179-5292-45ec-b0d1-d7f0ab8b30a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:35.123207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab5179-5292-45ec-b0d1-d7f0ab8b30a3 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:35.132959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 489/992] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 05:25:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:35.133999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53a72b-b815-4108-9567-58f810e27989 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:35.163300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:35.235601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53a72b-b815-4108-9567-58f810e27989 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:35.269544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f313d-0ac6-4aef-847b-6d90c4ec423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:35.270177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f313d-0ac6-4aef-847b-6d90c4ec423f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:35.270454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f313d-0ac6-4aef-847b-6d90c4ec423f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:35.270721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f313d-0ac6-4aef-847b-6d90c4ec423f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:35.271425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f313d-0ac6-4aef-847b-6d90c4ec423f None admin] RBAC: Adding query filter params (interface=admin, service_id=13770568bf9e4296b34a1c10f3164d46, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:35.271734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f313d-0ac6-4aef-847b-6d90c4ec423f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:35.273107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f313d-0ac6-4aef-847b-6d90c4ec423f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:35.283837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 504/993] 10.222.0.26 () {56 vars in 1249 bytes} [Tue Oct 4 05:25:35 2022] GET /identity/v3/endpoints?service_id=13770568bf9e4296b34a1c10f3164d46&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:35.284699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab5179-5292-45ec-b0d1-d7f0ab8b30a3 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:35.306676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:35.385926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab5179-5292-45ec-b0d1-d7f0ab8b30a3 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:35.420358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf8e43-0892-47ed-9136-333e616bb4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:35.420997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf8e43-0892-47ed-9136-333e616bb4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:35.421277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf8e43-0892-47ed-9136-333e616bb4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:35.421548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf8e43-0892-47ed-9136-333e616bb4f8 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:35.422174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf8e43-0892-47ed-9136-333e616bb4f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:35.423527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf8e43-0892-47ed-9136-333e616bb4f8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:35.430505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 490/994] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 05:25:35 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:25:36.705102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd4da5-62c4-4b72-b11c-054133df2d6b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:36.705795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd4da5-62c4-4b72-b11c-054133df2d6b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:36.705795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd4da5-62c4-4b72-b11c-054133df2d6b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:36.707063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 505/995] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:36.713115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7ecbf-8e29-48f9-b737-3cf35a15e867 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:36.713623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7ecbf-8e29-48f9-b737-3cf35a15e867 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:36.713725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7ecbf-8e29-48f9-b737-3cf35a15e867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:36.732214 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-19a7ecbf-8e29-48f9-b737-3cf35a15e867 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:36.733417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-19a7ecbf-8e29-48f9-b737-3cf35a15e867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:36.806300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 491/996] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:36.812222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e95f38-793c-4686-88c3-3a0a3e64a946 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:36.812512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e95f38-793c-4686-88c3-3a0a3e64a946 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:36.812791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e95f38-793c-4686-88c3-3a0a3e64a946 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:36.814852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 506/997] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:36.819440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539b4e9-c502-4380-a17f-6ba7a87530a3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:36.819732 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539b4e9-c502-4380-a17f-6ba7a87530a3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:36.820010 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539b4e9-c502-4380-a17f-6ba7a87530a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:36.838395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d539b4e9-c502-4380-a17f-6ba7a87530a3 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:36.839579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d539b4e9-c502-4380-a17f-6ba7a87530a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:36.849835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:36.912443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 492/998] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:37.071448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e95f38-793c-4686-88c3-3a0a3e64a946 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:37.164000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:37.173725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e95f38-793c-4686-88c3-3a0a3e64a946 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:37.182395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539b4e9-c502-4380-a17f-6ba7a87530a3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:37.212612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b526fbe-8ece-40a9-8f25-0b27ccbf1e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:37.213566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b526fbe-8ece-40a9-8f25-0b27ccbf1e34 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:37.214082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b526fbe-8ece-40a9-8f25-0b27ccbf1e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:37.214399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b526fbe-8ece-40a9-8f25-0b27ccbf1e34 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:37.215102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b526fbe-8ece-40a9-8f25-0b27ccbf1e34 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:37.216526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b526fbe-8ece-40a9-8f25-0b27ccbf1e34 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:37.282027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539b4e9-c502-4380-a17f-6ba7a87530a3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:37.302713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 507/999] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:25:37 2022] POST /identity/v3/services => generated 242 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:37.331249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876e6e4-5041-48e1-97b0-37655e5843e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:37.331829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876e6e4-5041-48e1-97b0-37655e5843e8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:37.332155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876e6e4-5041-48e1-97b0-37655e5843e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:37.332430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876e6e4-5041-48e1-97b0-37655e5843e8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:37.333024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876e6e4-5041-48e1-97b0-37655e5843e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:37.334433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876e6e4-5041-48e1-97b0-37655e5843e8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f876e6e4-5041-48e1-97b0-37655e5843e8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:37.344672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:37.345778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:37.346847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 493/1000] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:37.351763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b526fbe-8ece-40a9-8f25-0b27ccbf1e34 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:37.464733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b526fbe-8ece-40a9-8f25-0b27ccbf1e34 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:37.504820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e9a74-dfdd-4ee0-a5dd-58d39dbbabda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:37.505435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e9a74-dfdd-4ee0-a5dd-58d39dbbabda None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:37.505718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e9a74-dfdd-4ee0-a5dd-58d39dbbabda None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:37.505996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e9a74-dfdd-4ee0-a5dd-58d39dbbabda None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:37.506663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e9a74-dfdd-4ee0-a5dd-58d39dbbabda None admin] RBAC: Adding query filter params (name=compute) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:37.506972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e9a74-dfdd-4ee0-a5dd-58d39dbbabda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:37.508337 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e9a74-dfdd-4ee0-a5dd-58d39dbbabda None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:37.513504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 508/1001] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:37.519440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876e6e4-5041-48e1-97b0-37655e5843e8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:37.523842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:37.583941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e621363-0240-4f24-a92b-ca3c9a8231a1 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:37.584127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e621363-0240-4f24-a92b-ca3c9a8231a1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:37.584430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e621363-0240-4f24-a92b-ca3c9a8231a1 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:37.585938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 509/1002] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:37.590801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73817be4-e3a6-4c2e-b32e-f2467bfed6fa None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:37.591083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73817be4-e3a6-4c2e-b32e-f2467bfed6fa None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:37.591356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73817be4-e3a6-4c2e-b32e-f2467bfed6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:37.609738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-73817be4-e3a6-4c2e-b32e-f2467bfed6fa None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:37.610956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-73817be4-e3a6-4c2e-b32e-f2467bfed6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:37.621853 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876e6e4-5041-48e1-97b0-37655e5843e8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:37.622234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:37.658166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a1ee-f62c-4970-9d67-f6ef2dacf88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:37.658789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a1ee-f62c-4970-9d67-f6ef2dacf88e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:37.659062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a1ee-f62c-4970-9d67-f6ef2dacf88e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:37.659327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a1ee-f62c-4970-9d67-f6ef2dacf88e None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:37.659989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a1ee-f62c-4970-9d67-f6ef2dacf88e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:37.660289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a1ee-f62c-4970-9d67-f6ef2dacf88e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:37.661636 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a1ee-f62c-4970-9d67-f6ef2dacf88e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:37.666987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 494/1003] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:37.673238 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a1ee-f62c-4970-9d67-f6ef2dacf88e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:37.682359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 510/1004] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:37 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:37.740892 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:37.773985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a1ee-f62c-4970-9d67-f6ef2dacf88e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:37.808354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81efbf-b454-43c8-8d54-67143843e415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:37.808953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81efbf-b454-43c8-8d54-67143843e415 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:37.809230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81efbf-b454-43c8-8d54-67143843e415 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:37.809503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81efbf-b454-43c8-8d54-67143843e415 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:37.810226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81efbf-b454-43c8-8d54-67143843e415 None admin] RBAC: Adding query filter params (interface=public, service_id=bb2f2d40075a4d368c541a16883c5495, region_id=RegionOne) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:37.810532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81efbf-b454-43c8-8d54-67143843e415 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:37.811944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81efbf-b454-43c8-8d54-67143843e415 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:37.817218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 495/1005] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 05:25:37 2022] GET /identity/v3/endpoints?service_id=bb2f2d40075a4d368c541a16883c5495&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:37.823164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73817be4-e3a6-4c2e-b32e-f2467bfed6fa None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:37.926799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73817be4-e3a6-4c2e-b32e-f2467bfed6fa None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:37.955403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81efbf-b454-43c8-8d54-67143843e415 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:37.960138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6677a39-eb7f-4cac-bc79-08d83b8e4478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:37.960746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6677a39-eb7f-4cac-bc79-08d83b8e4478 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:37.961020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6677a39-eb7f-4cac-bc79-08d83b8e4478 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:37.961288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6677a39-eb7f-4cac-bc79-08d83b8e4478 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:37.961891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6677a39-eb7f-4cac-bc79-08d83b8e4478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:37.963273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6677a39-eb7f-4cac-bc79-08d83b8e4478 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:37.978569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 511/1006] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 05:25:37 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:25:38.065300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81efbf-b454-43c8-8d54-67143843e415 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:38.096561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:38.102619 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e0b13-3575-43a3-afce-12e4f0d08af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:38.103270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0b13-3575-43a3-afce-12e4f0d08af6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:38.103543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0b13-3575-43a3-afce-12e4f0d08af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:38.103808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0b13-3575-43a3-afce-12e4f0d08af6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:38.106805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e0b13-3575-43a3-afce-12e4f0d08af6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:38.106805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e0b13-3575-43a3-afce-12e4f0d08af6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3d5e0b13-3575-43a3-afce-12e4f0d08af6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:38.115779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:38.116872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 05:25:38.117785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:38.122196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 496/1007] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 05:25:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:38.123338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6677a39-eb7f-4cac-bc79-08d83b8e4478 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:38.138127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:38.229371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6677a39-eb7f-4cac-bc79-08d83b8e4478 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:38.262131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d1885-ab0c-4f15-ba6c-04aea4a0ce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:38.262758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d1885-ab0c-4f15-ba6c-04aea4a0ce63 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:38.263033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d1885-ab0c-4f15-ba6c-04aea4a0ce63 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:38.263303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d1885-ab0c-4f15-ba6c-04aea4a0ce63 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:38.263919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d1885-ab0c-4f15-ba6c-04aea4a0ce63 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:38.264220 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d1885-ab0c-4f15-ba6c-04aea4a0ce63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:38.266098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d1885-ab0c-4f15-ba6c-04aea4a0ce63 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:38.271049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 512/1008] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 05:25:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:38.276524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e0b13-3575-43a3-afce-12e4f0d08af6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:38.379545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e0b13-3575-43a3-afce-12e4f0d08af6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:38.413472 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701fd48-b3b6-4575-94ff-532e5ba9ce48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:38.414131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701fd48-b3b6-4575-94ff-532e5ba9ce48 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:38.414435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701fd48-b3b6-4575-94ff-532e5ba9ce48 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:38.414706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701fd48-b3b6-4575-94ff-532e5ba9ce48 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:38.415331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701fd48-b3b6-4575-94ff-532e5ba9ce48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:38.415635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701fd48-b3b6-4575-94ff-532e5ba9ce48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:38.416985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701fd48-b3b6-4575-94ff-532e5ba9ce48 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:38.426202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 497/1009] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 05:25:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:38.428022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d1885-ab0c-4f15-ba6c-04aea4a0ce63 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:38.504984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:38.529617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d1885-ab0c-4f15-ba6c-04aea4a0ce63 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:38.564125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12793ae-2588-46dd-ba32-f7f10946190c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:38.564743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12793ae-2588-46dd-ba32-f7f10946190c None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:38.565020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12793ae-2588-46dd-ba32-f7f10946190c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:38.565295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12793ae-2588-46dd-ba32-f7f10946190c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:38.565953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12793ae-2588-46dd-ba32-f7f10946190c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:38.567348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12793ae-2588-46dd-ba32-f7f10946190c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:38.651228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 513/1010] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:25:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:39.497949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60446d8a-1665-4fdf-8a15-86c8b87ffd7a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:39.498503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60446d8a-1665-4fdf-8a15-86c8b87ffd7a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:39.498605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60446d8a-1665-4fdf-8a15-86c8b87ffd7a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:39.499951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 498/1011] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:39.505089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ae998-28c5-434c-8859-bb9bdf0fe3ec None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:39.505468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ae998-28c5-434c-8859-bb9bdf0fe3ec None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:39.505636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ae998-28c5-434c-8859-bb9bdf0fe3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:39.524990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bc8ae998-28c5-434c-8859-bb9bdf0fe3ec None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:39.526267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bc8ae998-28c5-434c-8859-bb9bdf0fe3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:39.601778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 514/1012] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:39 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:39.871851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60446d8a-1665-4fdf-8a15-86c8b87ffd7a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:40.005650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:40.051383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60446d8a-1665-4fdf-8a15-86c8b87ffd7a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:40.086394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f478538-9a5d-4137-b27e-123090f5a530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:40.087024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f478538-9a5d-4137-b27e-123090f5a530 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.087305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f478538-9a5d-4137-b27e-123090f5a530 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.087578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f478538-9a5d-4137-b27e-123090f5a530 None admin] PATH_INFO: `/v3/services/image` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.088157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f478538-9a5d-4137-b27e-123090f5a530 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:40.089531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f478538-9a5d-4137-b27e-123090f5a530 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5f478538-9a5d-4137-b27e-123090f5a530 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:40.098335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:40.099317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 05:25:40.100274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:40.101672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 499/1013] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Oct 4 05:25:39 2022] GET /identity/v3/services/image => generated 86 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:40.105314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ae998-28c5-434c-8859-bb9bdf0fe3ec None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:40.145443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ce0d6-ad15-4f54-baae-7ff3c89d6c12 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.145718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ce0d6-ad15-4f54-baae-7ff3c89d6c12 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.145912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ce0d6-ad15-4f54-baae-7ff3c89d6c12 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.147412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 500/1014] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:40.152167 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bf53f-b902-481b-82b0-2ba98d110c22 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.152456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bf53f-b902-481b-82b0-2ba98d110c22 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.152746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bf53f-b902-481b-82b0-2ba98d110c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.166626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:40.172210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-248bf53f-b902-481b-82b0-2ba98d110c22 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:40.173395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-248bf53f-b902-481b-82b0-2ba98d110c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:40.208639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ae998-28c5-434c-8859-bb9bdf0fe3ec None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:40.243244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 501/1015] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:40 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:40.248620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3581b-f567-4e02-860c-be4bf8c54e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:40.249788 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3581b-f567-4e02-860c-be4bf8c54e6e None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.250131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3581b-f567-4e02-860c-be4bf8c54e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.250508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3581b-f567-4e02-860c-be4bf8c54e6e None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.251200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3581b-f567-4e02-860c-be4bf8c54e6e None admin] RBAC: Adding query filter params (name=image) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:40.251564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3581b-f567-4e02-860c-be4bf8c54e6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:40.252992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3581b-f567-4e02-860c-be4bf8c54e6e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:40.262628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 515/1016] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:40.263827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bf53f-b902-481b-82b0-2ba98d110c22 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:40.319360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:40.363913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bf53f-b902-481b-82b0-2ba98d110c22 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:40.397617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b385b-3265-4cc4-8473-ae010951ef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:40.398126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b385b-3265-4cc4-8473-ae010951ef97 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.398428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b385b-3265-4cc4-8473-ae010951ef97 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.398703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b385b-3265-4cc4-8473-ae010951ef97 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.399320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b385b-3265-4cc4-8473-ae010951ef97 None admin] RBAC: Adding query filter params (type=image) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:40.399622 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b385b-3265-4cc4-8473-ae010951ef97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:40.400959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b385b-3265-4cc4-8473-ae010951ef97 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:40.410491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 502/1017] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:40.411563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3581b-f567-4e02-860c-be4bf8c54e6e None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:40.510601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3581b-f567-4e02-860c-be4bf8c54e6e None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:40.517539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b385b-3265-4cc4-8473-ae010951ef97 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:40.534923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:40.547375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60668721-5957-422d-81c4-929f974730de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:40.547989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668721-5957-422d-81c4-929f974730de None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.548267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668721-5957-422d-81c4-929f974730de None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.548539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668721-5957-422d-81c4-929f974730de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.549235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60668721-5957-422d-81c4-929f974730de None admin] RBAC: Adding query filter params (interface=public, service_id=82b1d68aba184377b21472e61f61d22c, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:40.549544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60668721-5957-422d-81c4-929f974730de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:40.550928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60668721-5957-422d-81c4-929f974730de None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:40.560423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 516/1018] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 05:25:40 2022] GET /identity/v3/endpoints?service_id=82b1d68aba184377b21472e61f61d22c&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:40.562017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60668721-5957-422d-81c4-929f974730de None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:40.623774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b385b-3265-4cc4-8473-ae010951ef97 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:40.642208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:40.660212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136b47d-f07b-44ff-b7d8-21c663da529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:40.660808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136b47d-f07b-44ff-b7d8-21c663da529c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.661108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136b47d-f07b-44ff-b7d8-21c663da529c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.661314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60668721-5957-422d-81c4-929f974730de None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:40.661435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136b47d-f07b-44ff-b7d8-21c663da529c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.661941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136b47d-f07b-44ff-b7d8-21c663da529c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:40.664268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136b47d-f07b-44ff-b7d8-21c663da529c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b136b47d-f07b-44ff-b7d8-21c663da529c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:40.675830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:40.676895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 05:25:40.677851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:40.682380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 503/1019] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:40.683340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136b47d-f07b-44ff-b7d8-21c663da529c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:40.699975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1919f50-c0f0-4e4c-a7e7-7bb9e6772e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:40.700612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1919f50-c0f0-4e4c-a7e7-7bb9e6772e83 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.700889 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1919f50-c0f0-4e4c-a7e7-7bb9e6772e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.701157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1919f50-c0f0-4e4c-a7e7-7bb9e6772e83 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.701776 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1919f50-c0f0-4e4c-a7e7-7bb9e6772e83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:40.703262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1919f50-c0f0-4e4c-a7e7-7bb9e6772e83 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:40.718185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 517/1020] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 05:25:40 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:25:40.803547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136b47d-f07b-44ff-b7d8-21c663da529c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:40.810075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fa91c-0c03-42e0-8b6e-d618f0255d8f None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.810421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fa91c-0c03-42e0-8b6e-d618f0255d8f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.810663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fa91c-0c03-42e0-8b6e-d618f0255d8f None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.813063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 518/1021] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:40.818781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dcaed-b364-409d-b71b-cef6c198b938 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.819321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dcaed-b364-409d-b71b-cef6c198b938 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.819599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dcaed-b364-409d-b71b-cef6c198b938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.838515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e54dcaed-b364-409d-b71b-cef6c198b938 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:40.839765 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e54dcaed-b364-409d-b71b-cef6c198b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:40.841393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26e224-5040-4401-940f-37960b052963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:40.842020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26e224-5040-4401-940f-37960b052963 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:40.842335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26e224-5040-4401-940f-37960b052963 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:40.843152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26e224-5040-4401-940f-37960b052963 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:40.843775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26e224-5040-4401-940f-37960b052963 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:40.844077 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26e224-5040-4401-940f-37960b052963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:40.845475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26e224-5040-4401-940f-37960b052963 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:40.850941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 504/1022] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:40.856910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26e224-5040-4401-940f-37960b052963 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:40.900006 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:40.909586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:40.924680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 519/1023] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:40 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:40.965963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26e224-5040-4401-940f-37960b052963 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:41.000276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af41bb1-b2d9-4110-b948-7a49e124b761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:41.000880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af41bb1-b2d9-4110-b948-7a49e124b761 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:41.001154 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af41bb1-b2d9-4110-b948-7a49e124b761 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:41.001428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af41bb1-b2d9-4110-b948-7a49e124b761 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:41.002044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af41bb1-b2d9-4110-b948-7a49e124b761 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:41.002372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af41bb1-b2d9-4110-b948-7a49e124b761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:41.003722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af41bb1-b2d9-4110-b948-7a49e124b761 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:41.013159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 505/1024] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 05:25:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:41.014311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dcaed-b364-409d-b71b-cef6c198b938 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:41.112388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dcaed-b364-409d-b71b-cef6c198b938 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:41.146262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa289f7-9d3d-40fc-be12-01c13cb954aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:41.146979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa289f7-9d3d-40fc-be12-01c13cb954aa None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:41.147352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa289f7-9d3d-40fc-be12-01c13cb954aa None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:41.147716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa289f7-9d3d-40fc-be12-01c13cb954aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:41.148483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa289f7-9d3d-40fc-be12-01c13cb954aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:41.149936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa289f7-9d3d-40fc-be12-01c13cb954aa None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:41.195953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af41bb1-b2d9-4110-b948-7a49e124b761 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:41.248775 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 520/1025] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:25:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:25:41.313520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af41bb1-b2d9-4110-b948-7a49e124b761 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:41.325268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:41.357345 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77429212-6571-42b4-9928-603ccecbb63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:41.358093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77429212-6571-42b4-9928-603ccecbb63f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:41.358875 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77429212-6571-42b4-9928-603ccecbb63f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:41.359498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77429212-6571-42b4-9928-603ccecbb63f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:41.361142 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77429212-6571-42b4-9928-603ccecbb63f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:41.362745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77429212-6571-42b4-9928-603ccecbb63f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:41.366187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 506/1026] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:41.370815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa289f7-9d3d-40fc-be12-01c13cb954aa None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:41.414167 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:41.482364 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa289f7-9d3d-40fc-be12-01c13cb954aa None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:41.519072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa31c4-f2c9-4b94-8a61-4277688506ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:41.519684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa31c4-f2c9-4b94-8a61-4277688506ff None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:41.519960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa31c4-f2c9-4b94-8a61-4277688506ff None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:41.520235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa31c4-f2c9-4b94-8a61-4277688506ff None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:41.520900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa31c4-f2c9-4b94-8a61-4277688506ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:41.522369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa31c4-f2c9-4b94-8a61-4277688506ff None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:41.606487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 521/1027] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:25:41 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:25:42.899891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a8ccf-bf7d-4057-b810-1f144af281ba None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:42.900563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a8ccf-bf7d-4057-b810-1f144af281ba None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:42.900563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a8ccf-bf7d-4057-b810-1f144af281ba None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:42.901945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 507/1028] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:42.907114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeda8a-cde7-456e-935e-3186c69eb30d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:42.907515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeda8a-cde7-456e-935e-3186c69eb30d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:42.907680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeda8a-cde7-456e-935e-3186c69eb30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:42.927172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a9aeda8a-cde7-456e-935e-3186c69eb30d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:42.928367 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a9aeda8a-cde7-456e-935e-3186c69eb30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:42.998302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 522/1029] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:43.271491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a8ccf-bf7d-4057-b810-1f144af281ba None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:43.374152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a8ccf-bf7d-4057-b810-1f144af281ba None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:43.411899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91ecc0-7fd9-45cb-8bfb-f7c9d24bfcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:43.412558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ecc0-7fd9-45cb-8bfb-f7c9d24bfcde None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:43.412838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ecc0-7fd9-45cb-8bfb-f7c9d24bfcde None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:43.413112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91ecc0-7fd9-45cb-8bfb-f7c9d24bfcde None admin] PATH_INFO: `/v3/services/image` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:43.413707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91ecc0-7fd9-45cb-8bfb-f7c9d24bfcde None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:43.415138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91ecc0-7fd9-45cb-8bfb-f7c9d24bfcde None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-cc91ecc0-7fd9-45cb-8bfb-f7c9d24bfcde None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:43.424873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:43.425931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:43.426835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 508/1030] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Oct 4 05:25:43 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:43.433723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aeda8a-cde7-456e-935e-3186c69eb30d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:43.455458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:43.507908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a5eca-c960-4d4f-9155-2ac455cbb687 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:43.509842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a5eca-c960-4d4f-9155-2ac455cbb687 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:43.509842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a5eca-c960-4d4f-9155-2ac455cbb687 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:43.509842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 509/1031] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:43.515461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a32781-caf3-40a3-af19-fa54f2900841 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:43.515745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a32781-caf3-40a3-af19-fa54f2900841 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:43.516016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a32781-caf3-40a3-af19-fa54f2900841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:43.534739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-82a32781-caf3-40a3-af19-fa54f2900841 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:43.535950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-82a32781-caf3-40a3-af19-fa54f2900841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:43.538059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aeda8a-cde7-456e-935e-3186c69eb30d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:43.568522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:43.578497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b72e2c-da7c-45b9-a6d5-25ea5d0d7cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:43.579121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b72e2c-da7c-45b9-a6d5-25ea5d0d7cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:43.579400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b72e2c-da7c-45b9-a6d5-25ea5d0d7cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:43.583987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b72e2c-da7c-45b9-a6d5-25ea5d0d7cb3 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:43.583987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b72e2c-da7c-45b9-a6d5-25ea5d0d7cb3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:43.583987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b72e2c-da7c-45b9-a6d5-25ea5d0d7cb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:43.583987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b72e2c-da7c-45b9-a6d5-25ea5d0d7cb3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:43.586819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 523/1032] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:43.592382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b72e2c-da7c-45b9-a6d5-25ea5d0d7cb3 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:43.612747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 510/1033] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:43.680408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:43.698033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b72e2c-da7c-45b9-a6d5-25ea5d0d7cb3 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:43.734951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef0c6c-3366-462c-a4d8-8b0a2c4086a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:43.735559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0c6c-3366-462c-a4d8-8b0a2c4086a0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:43.735843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0c6c-3366-462c-a4d8-8b0a2c4086a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:43.736118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef0c6c-3366-462c-a4d8-8b0a2c4086a0 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:43.736743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef0c6c-3366-462c-a4d8-8b0a2c4086a0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:43.737049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef0c6c-3366-462c-a4d8-8b0a2c4086a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:43.738444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef0c6c-3366-462c-a4d8-8b0a2c4086a0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:43.748111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 524/1034] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:43.749447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a32781-caf3-40a3-af19-fa54f2900841 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:43.797021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ffb9a-da24-4ad9-8155-dbfd9d39865c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:43.797309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ffb9a-da24-4ad9-8155-dbfd9d39865c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:43.797502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ffb9a-da24-4ad9-8155-dbfd9d39865c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:43.799004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 525/1035] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:43.803782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612b58d-901c-4d4f-89e8-7172c3334231 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:43.804068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612b58d-901c-4d4f-89e8-7172c3334231 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:43.804340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612b58d-901c-4d4f-89e8-7172c3334231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:43.822896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e612b58d-901c-4d4f-89e8-7172c3334231 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:43.824087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e612b58d-901c-4d4f-89e8-7172c3334231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:43.851199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a32781-caf3-40a3-af19-fa54f2900841 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:43.887291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe93a1-33fa-4e26-ad6e-57dd11224eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:43.887913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe93a1-33fa-4e26-ad6e-57dd11224eca None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:43.888189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe93a1-33fa-4e26-ad6e-57dd11224eca None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:43.888468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe93a1-33fa-4e26-ad6e-57dd11224eca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:43.889140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe93a1-33fa-4e26-ad6e-57dd11224eca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:43.891150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe93a1-33fa-4e26-ad6e-57dd11224eca None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:43.898849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 526/1036] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:43.899851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612b58d-901c-4d4f-89e8-7172c3334231 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:43.975506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:43.996645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612b58d-901c-4d4f-89e8-7172c3334231 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:44.029416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3472500-296f-44d3-83a8-3bcd6053b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:44.030718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3472500-296f-44d3-83a8-3bcd6053b4df None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:44.030997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3472500-296f-44d3-83a8-3bcd6053b4df None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:44.031267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3472500-296f-44d3-83a8-3bcd6053b4df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:44.041208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3472500-296f-44d3-83a8-3bcd6053b4df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f3472500-296f-44d3-83a8-3bcd6053b4df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:44.046660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:44.053681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 527/1037] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 05:25:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:44.054631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3472500-296f-44d3-83a8-3bcd6053b4df None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:44.126592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 511/1038] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:25:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:25:44.165926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3472500-296f-44d3-83a8-3bcd6053b4df None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:44.178697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe93a1-33fa-4e26-ad6e-57dd11224eca None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:44.206648 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:44.215562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da435d99-3094-4c8e-b389-bff0351300bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:44.221898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da435d99-3094-4c8e-b389-bff0351300bd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:44.221898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da435d99-3094-4c8e-b389-bff0351300bd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:44.221898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da435d99-3094-4c8e-b389-bff0351300bd None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:44.221898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da435d99-3094-4c8e-b389-bff0351300bd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:44.221898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da435d99-3094-4c8e-b389-bff0351300bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:44.221898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da435d99-3094-4c8e-b389-bff0351300bd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:44.223333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:44.231132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 528/1039] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 05:25:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:44.237083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da435d99-3094-4c8e-b389-bff0351300bd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:44.298157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe93a1-33fa-4e26-ad6e-57dd11224eca None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:44.340135 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6135-4268-437e-b265-ee76f3a34d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:44.340690 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6135-4268-437e-b265-ee76f3a34d62 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:44.340965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6135-4268-437e-b265-ee76f3a34d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:44.341233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c6135-4268-437e-b265-ee76f3a34d62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:44.342488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c6135-4268-437e-b265-ee76f3a34d62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:44.344867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c6135-4268-437e-b265-ee76f3a34d62 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:44.346487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 512/1040] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:44.352730 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6135-4268-437e-b265-ee76f3a34d62 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:44.357094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da435d99-3094-4c8e-b389-bff0351300bd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:44.391701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18b4c1-a872-487b-b128-d53232921d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:44.392339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18b4c1-a872-487b-b128-d53232921d74 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:44.392616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18b4c1-a872-487b-b128-d53232921d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:44.392883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18b4c1-a872-487b-b128-d53232921d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:44.403935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18b4c1-a872-487b-b128-d53232921d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:44.405938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18b4c1-a872-487b-b128-d53232921d74 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ae18b4c1-a872-487b-b128-d53232921d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:44.420455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:44.421479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:44.422429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:44.422429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:44.422429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:44.422429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 529/1041] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:44.427179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18b4c1-a872-487b-b128-d53232921d74 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:44.449947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:44.460066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c6135-4268-437e-b265-ee76f3a34d62 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:44.477089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:44.499259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99997e63-ea4b-4d4b-a0c1-e392e742be4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:44.500288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99997e63-ea4b-4d4b-a0c1-e392e742be4d None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:44.500729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99997e63-ea4b-4d4b-a0c1-e392e742be4d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:44.501149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99997e63-ea4b-4d4b-a0c1-e392e742be4d None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:44.501958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99997e63-ea4b-4d4b-a0c1-e392e742be4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:44.503569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99997e63-ea4b-4d4b-a0c1-e392e742be4d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:44.528310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18b4c1-a872-487b-b128-d53232921d74 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:44.560395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208e45b-11b0-4957-9a87-ae80964c9ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:44.560996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208e45b-11b0-4957-9a87-ae80964c9ffb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:44.561806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208e45b-11b0-4957-9a87-ae80964c9ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:44.562075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208e45b-11b0-4957-9a87-ae80964c9ffb None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:44.562733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208e45b-11b0-4957-9a87-ae80964c9ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:44.563036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208e45b-11b0-4957-9a87-ae80964c9ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:44.564389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208e45b-11b0-4957-9a87-ae80964c9ffb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:44.576022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 530/1042] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:44.582269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208e45b-11b0-4957-9a87-ae80964c9ffb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:44.616235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 513/1043] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:25:44 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:25:44.690902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208e45b-11b0-4957-9a87-ae80964c9ffb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:44.701604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:44.728738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae2ff3-2a60-4390-9200-f9cd6b1d16eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:44.729471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae2ff3-2a60-4390-9200-f9cd6b1d16eb None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:44.729814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae2ff3-2a60-4390-9200-f9cd6b1d16eb None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:44.730167 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae2ff3-2a60-4390-9200-f9cd6b1d16eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:44.738213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae2ff3-2a60-4390-9200-f9cd6b1d16eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:44.739741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae2ff3-2a60-4390-9200-f9cd6b1d16eb None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-a8ae2ff3-2a60-4390-9200-f9cd6b1d16eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:44.753017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:44.754413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:44.755574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:44.755574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:44.755574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:44.755574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:44.755574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:44.755574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:44.755574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:44.755574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:44.755574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:25:44.755574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:44.756208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 531/1044] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 05:25:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:44.761278 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99997e63-ea4b-4d4b-a0c1-e392e742be4d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:44.871953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99997e63-ea4b-4d4b-a0c1-e392e742be4d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:44.911594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead2ab8-be98-490a-a4e2-4743a8f181ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:44.912203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead2ab8-be98-490a-a4e2-4743a8f181ce None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:44.912478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead2ab8-be98-490a-a4e2-4743a8f181ce None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:44.912749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead2ab8-be98-490a-a4e2-4743a8f181ce None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:44.913471 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead2ab8-be98-490a-a4e2-4743a8f181ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:44.913772 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead2ab8-be98-490a-a4e2-4743a8f181ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:44.915205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead2ab8-be98-490a-a4e2-4743a8f181ce None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:44.933356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 514/1045] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 05:25:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:44.939666 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae2ff3-2a60-4390-9200-f9cd6b1d16eb None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:45.040426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae2ff3-2a60-4390-9200-f9cd6b1d16eb None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:45.074632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90bd11-f62b-48d1-ad58-af639adf4630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:45.075246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90bd11-f62b-48d1-ad58-af639adf4630 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:45.075519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90bd11-f62b-48d1-ad58-af639adf4630 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:45.075785 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90bd11-f62b-48d1-ad58-af639adf4630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:45.108861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:45.140853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90bd11-f62b-48d1-ad58-af639adf4630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:45.142247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90bd11-f62b-48d1-ad58-af639adf4630 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:45.243758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 532/1046] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 05:25:45.245110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead2ab8-be98-490a-a4e2-4743a8f181ce None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:45.255339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:45.345696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead2ab8-be98-490a-a4e2-4743a8f181ce None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:45.379286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7b127-a8ee-4948-a53c-a0990ea2707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:45.379917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7b127-a8ee-4948-a53c-a0990ea2707d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:45.380191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7b127-a8ee-4948-a53c-a0990ea2707d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:45.380461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7b127-a8ee-4948-a53c-a0990ea2707d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:45.391169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7b127-a8ee-4948-a53c-a0990ea2707d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:45.392524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7b127-a8ee-4948-a53c-a0990ea2707d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-d2c7b127-a8ee-4948-a53c-a0990ea2707d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:45.406946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:45.407915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:45.413566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:45.413566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:45.413566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:45.413566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 515/1047] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:45.413810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90bd11-f62b-48d1-ad58-af639adf4630 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:45.479374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:45.513173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90bd11-f62b-48d1-ad58-af639adf4630 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:45.547741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ff557-72a4-4d46-a19e-fde529b5e5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:45.548348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ff557-72a4-4d46-a19e-fde529b5e5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:45.548624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ff557-72a4-4d46-a19e-fde529b5e5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:45.548894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ff557-72a4-4d46-a19e-fde529b5e5a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:45.549526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ff557-72a4-4d46-a19e-fde529b5e5a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:45.549828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ff557-72a4-4d46-a19e-fde529b5e5a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:45.551210 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ff557-72a4-4d46-a19e-fde529b5e5a6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:45.567010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 533/1048] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:45.568199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7b127-a8ee-4948-a53c-a0990ea2707d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:45.628186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:45.668480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7b127-a8ee-4948-a53c-a0990ea2707d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:45.701624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459e83c-6123-4e96-8417-2973d6db49f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:45.702911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459e83c-6123-4e96-8417-2973d6db49f8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:45.703280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459e83c-6123-4e96-8417-2973d6db49f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:45.703641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459e83c-6123-4e96-8417-2973d6db49f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:45.710165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459e83c-6123-4e96-8417-2973d6db49f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:45.711704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459e83c-6123-4e96-8417-2973d6db49f8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7459e83c-6123-4e96-8417-2973d6db49f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:45.723298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:45.724356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:45.725432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:45.725432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:45.725432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:45.725432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:45.725432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:45.725432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:45.725432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:45.725432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:45.732881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 516/1049] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:45.733042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ff557-72a4-4d46-a19e-fde529b5e5a6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:45.833527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ff557-72a4-4d46-a19e-fde529b5e5a6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:45.860631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:45.869005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da3ff2-7fa3-4b16-b6f0-88aaaa689385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:45.869685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da3ff2-7fa3-4b16-b6f0-88aaaa689385 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:45.870029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da3ff2-7fa3-4b16-b6f0-88aaaa689385 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:45.870391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da3ff2-7fa3-4b16-b6f0-88aaaa689385 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:45.871146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da3ff2-7fa3-4b16-b6f0-88aaaa689385 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:45.871522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da3ff2-7fa3-4b16-b6f0-88aaaa689385 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:45.872999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da3ff2-7fa3-4b16-b6f0-88aaaa689385 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:45.890017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 534/1050] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:45.894808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459e83c-6123-4e96-8417-2973d6db49f8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:46.039110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459e83c-6123-4e96-8417-2973d6db49f8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:46.055475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:46.072723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda2b1c-2c13-40c0-8753-c8e4f7e0111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:46.073323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda2b1c-2c13-40c0-8753-c8e4f7e0111f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:46.073598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda2b1c-2c13-40c0-8753-c8e4f7e0111f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:46.073865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda2b1c-2c13-40c0-8753-c8e4f7e0111f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:46.074611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda2b1c-2c13-40c0-8753-c8e4f7e0111f None admin] RBAC: Adding query filter params (role.id=ad98bf0d9c9240e48f0243790aff940d, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=fba4f0473f4c4d3bb2d50b09f3de9837) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:46.074916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda2b1c-2c13-40c0-8753-c8e4f7e0111f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:46.076315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda2b1c-2c13-40c0-8753-c8e4f7e0111f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:46.082317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 517/1051] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:45 2022] GET /identity/v3/role_assignments?user.id=fba4f0473f4c4d3bb2d50b09f3de9837&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=ad98bf0d9c9240e48f0243790aff940d => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:46.315561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba38d87-2b19-443f-ae39-15c65501de32 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:46.316075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba38d87-2b19-443f-ae39-15c65501de32 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:46.316075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba38d87-2b19-443f-ae39-15c65501de32 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:46.317505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 535/1052] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:46.322933 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a73c57-0873-42d8-b7ea-26f894372991 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:46.323319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a73c57-0873-42d8-b7ea-26f894372991 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:46.323510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a73c57-0873-42d8-b7ea-26f894372991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:46.342072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-00a73c57-0873-42d8-b7ea-26f894372991 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:46.343364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-00a73c57-0873-42d8-b7ea-26f894372991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:46.410521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 518/1053] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:46.680943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba38d87-2b19-443f-ae39-15c65501de32 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:46.790163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba38d87-2b19-443f-ae39-15c65501de32 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:46.824296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c99d5-96ca-424a-96eb-28c5fa69159d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:46.824986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c99d5-96ca-424a-96eb-28c5fa69159d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:46.825266 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c99d5-96ca-424a-96eb-28c5fa69159d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:46.825589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c99d5-96ca-424a-96eb-28c5fa69159d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:46.866416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9580dee-b326-4cab-a8eb-de95b25084da None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:46.866744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9580dee-b326-4cab-a8eb-de95b25084da None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:46.866976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9580dee-b326-4cab-a8eb-de95b25084da None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:46.868437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 519/1054] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:46.873047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2a80e-6f40-4572-b9bd-f262ca76578e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:46.873330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2a80e-6f40-4572-b9bd-f262ca76578e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:46.874127 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2a80e-6f40-4572-b9bd-f262ca76578e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:46.882522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:46.894920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9dd2a80e-6f40-4572-b9bd-f262ca76578e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:46.896173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9dd2a80e-6f40-4572-b9bd-f262ca76578e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:46.909607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c99d5-96ca-424a-96eb-28c5fa69159d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:46.911120 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c99d5-96ca-424a-96eb-28c5fa69159d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:46.970712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 520/1055] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:47.014225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 536/1056] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:25:47.018792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2a80e-6f40-4572-b9bd-f262ca76578e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:47.062320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:47.118077 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2a80e-6f40-4572-b9bd-f262ca76578e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:47.151159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa96b1-f8ad-4e79-87a1-d55bba1a5d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:47.151782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa96b1-f8ad-4e79-87a1-d55bba1a5d59 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:47.152060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa96b1-f8ad-4e79-87a1-d55bba1a5d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:47.152333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa96b1-f8ad-4e79-87a1-d55bba1a5d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:47.162225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa96b1-f8ad-4e79-87a1-d55bba1a5d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:47.163646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa96b1-f8ad-4e79-87a1-d55bba1a5d59 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-acfa96b1-f8ad-4e79-87a1-d55bba1a5d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:47.178455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:47.179413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:47.180332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:47.180332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:47.180332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:47.180332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 521/1057] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:47.185845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c99d5-96ca-424a-96eb-28c5fa69159d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:47.244616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa96b1-f8ad-4e79-87a1-d55bba1a5d59 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:47.281921 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:47.293713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c99d5-96ca-424a-96eb-28c5fa69159d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:47.328804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314064d-900c-4fd7-b469-f39606370ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:47.329437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314064d-900c-4fd7-b469-f39606370ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:47.329668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314064d-900c-4fd7-b469-f39606370ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:47.329933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314064d-900c-4fd7-b469-f39606370ca6 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:47.330601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314064d-900c-4fd7-b469-f39606370ca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:47.330899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314064d-900c-4fd7-b469-f39606370ca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:47.332281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314064d-900c-4fd7-b469-f39606370ca6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:47.346182 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 537/1058] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:47.354069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:47.363259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa96b1-f8ad-4e79-87a1-d55bba1a5d59 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:47.400757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fed969-1e16-4257-8d75-67c7a125dd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:47.401415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fed969-1e16-4257-8d75-67c7a125dd87 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:47.401689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fed969-1e16-4257-8d75-67c7a125dd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:47.401961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fed969-1e16-4257-8d75-67c7a125dd87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:47.413407 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fed969-1e16-4257-8d75-67c7a125dd87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-59fed969-1e16-4257-8d75-67c7a125dd87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:47.418398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:47.419392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:47.420249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:47.422298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 522/1059] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:47.426265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314064d-900c-4fd7-b469-f39606370ca6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:47.535907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314064d-900c-4fd7-b469-f39606370ca6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:47.570738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7fb5b-ec49-4299-ab83-34a431929b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:47.571348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7fb5b-ec49-4299-ab83-34a431929b7c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:47.571623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7fb5b-ec49-4299-ab83-34a431929b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:47.571893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7fb5b-ec49-4299-ab83-34a431929b7c None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:47.572564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7fb5b-ec49-4299-ab83-34a431929b7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:47.572901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7fb5b-ec49-4299-ab83-34a431929b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:47.574280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7fb5b-ec49-4299-ab83-34a431929b7c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:47.583228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 538/1060] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:25:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:47.589402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fed969-1e16-4257-8d75-67c7a125dd87 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:47.697047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fed969-1e16-4257-8d75-67c7a125dd87 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:47.730261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cfb5b-bbed-4de9-b49e-88fcf6f6d0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:47.730881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cfb5b-bbed-4de9-b49e-88fcf6f6d0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:47.731158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cfb5b-bbed-4de9-b49e-88fcf6f6d0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:47.731426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618cfb5b-bbed-4de9-b49e-88fcf6f6d0d2 None admin] PATH_INFO: `/v3/users/897f2b46ebe441949f1a01ccf9b9bee0` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:47.744806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618cfb5b-bbed-4de9-b49e-88fcf6f6d0d2 None admin] RBAC: Authorizing `identity:get_user(user_id=897f2b46ebe441949f1a01ccf9b9bee0)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:47.746235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618cfb5b-bbed-4de9-b49e-88fcf6f6d0d2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:47.747832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 523/1061] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:25:47 2022] GET /identity/v3/users/897f2b46ebe441949f1a01ccf9b9bee0 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:47.753797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7fb5b-ec49-4299-ab83-34a431929b7c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:47.765891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:47.852068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7fb5b-ec49-4299-ab83-34a431929b7c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:47.888079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee2725-d1d4-48a2-b11b-d4ffcf89d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:47.888079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee2725-d1d4-48a2-b11b-d4ffcf89d635 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:47.888079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee2725-d1d4-48a2-b11b-d4ffcf89d635 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:47.888079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee2725-d1d4-48a2-b11b-d4ffcf89d635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:47.954185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee2725-d1d4-48a2-b11b-d4ffcf89d635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:47.955621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee2725-d1d4-48a2-b11b-d4ffcf89d635 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:47.966170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:48.049608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 539/1062] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:25:48.055810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cfb5b-bbed-4de9-b49e-88fcf6f6d0d2 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:48.070537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:48.156907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618cfb5b-bbed-4de9-b49e-88fcf6f6d0d2 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:48.188945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76d9b7-bcd4-4194-9c71-11efbfffee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:48.189571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76d9b7-bcd4-4194-9c71-11efbfffee3b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:48.189847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76d9b7-bcd4-4194-9c71-11efbfffee3b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:48.190115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76d9b7-bcd4-4194-9c71-11efbfffee3b None admin] PATH_INFO: `/v3/projects/1638d5fc33024482b7a64d56443f4c0c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:48.191240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76d9b7-bcd4-4194-9c71-11efbfffee3b None admin] RBAC: Authorizing `identity:get_project(project_id=1638d5fc33024482b7a64d56443f4c0c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:48.192639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76d9b7-bcd4-4194-9c71-11efbfffee3b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:48.194562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 524/1063] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:25:48 2022] GET /identity/v3/projects/1638d5fc33024482b7a64d56443f4c0c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:48.200965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee2725-d1d4-48a2-b11b-d4ffcf89d635 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:48.215873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a46e3-4e3a-4b50-93c0-aaa95e564be1 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:48.216154 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a46e3-4e3a-4b50-93c0-aaa95e564be1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:48.216425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a46e3-4e3a-4b50-93c0-aaa95e564be1 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:48.218370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 525/1064] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:48.223169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff4363-6356-4373-9006-a9ca30bb1d8b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:48.223447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff4363-6356-4373-9006-a9ca30bb1d8b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:48.223715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff4363-6356-4373-9006-a9ca30bb1d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:48.241117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5bff4363-6356-4373-9006-a9ca30bb1d8b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:48.242330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5bff4363-6356-4373-9006-a9ca30bb1d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:48.300870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee2725-d1d4-48a2-b11b-d4ffcf89d635 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:48.307133 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 526/1065] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:48.312185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:48.335060 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f90930-f987-4561-9b7e-dd48e4897a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:48.335696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f90930-f987-4561-9b7e-dd48e4897a5c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:48.335925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f90930-f987-4561-9b7e-dd48e4897a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:48.336194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f90930-f987-4561-9b7e-dd48e4897a5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:48.336912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f90930-f987-4561-9b7e-dd48e4897a5c None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=1638d5fc33024482b7a64d56443f4c0c, user.id=897f2b46ebe441949f1a01ccf9b9bee0) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:48.337218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f90930-f987-4561-9b7e-dd48e4897a5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:48.338667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f90930-f987-4561-9b7e-dd48e4897a5c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:48.343796 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 540/1066] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:48 2022] GET /identity/v3/role_assignments?user.id=897f2b46ebe441949f1a01ccf9b9bee0&scope.project.id=1638d5fc33024482b7a64d56443f4c0c&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:48.573440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff4363-6356-4373-9006-a9ca30bb1d8b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:48.585331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:48.669333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff4363-6356-4373-9006-a9ca30bb1d8b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:48.703024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de8f78-11ad-42dc-be3a-70fe3041599b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:48.703649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de8f78-11ad-42dc-be3a-70fe3041599b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:48.703926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de8f78-11ad-42dc-be3a-70fe3041599b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:48.704198 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de8f78-11ad-42dc-be3a-70fe3041599b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:48.713858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de8f78-11ad-42dc-be3a-70fe3041599b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-a5de8f78-11ad-42dc-be3a-70fe3041599b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:48.718710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:48.719839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:48.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 527/1067] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 05:25:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:48.725635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f90930-f987-4561-9b7e-dd48e4897a5c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:48.822885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f90930-f987-4561-9b7e-dd48e4897a5c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:48.854620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44ced7-f23b-4a52-bd01-53cb4e5312d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:48.855224 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44ced7-f23b-4a52-bd01-53cb4e5312d7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:48.855496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44ced7-f23b-4a52-bd01-53cb4e5312d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:48.855767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44ced7-f23b-4a52-bd01-53cb4e5312d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:48.856434 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44ced7-f23b-4a52-bd01-53cb4e5312d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:48.856731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44ced7-f23b-4a52-bd01-53cb4e5312d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:48.858657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44ced7-f23b-4a52-bd01-53cb4e5312d7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:48.866231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 541/1068] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 05:25:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:48.871807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de8f78-11ad-42dc-be3a-70fe3041599b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:48.943554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:48.973241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de8f78-11ad-42dc-be3a-70fe3041599b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:49.007005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1382f04-a46a-4e11-be84-786a61dc8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:49.007646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1382f04-a46a-4e11-be84-786a61dc8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:49.007903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1382f04-a46a-4e11-be84-786a61dc8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:49.008174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1382f04-a46a-4e11-be84-786a61dc8ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:49.019200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1382f04-a46a-4e11-be84-786a61dc8ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:49.020639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1382f04-a46a-4e11-be84-786a61dc8ee4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e1382f04-a46a-4e11-be84-786a61dc8ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:49.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:49.035812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:49.036880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:49.036880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:49.036880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:49.036880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 528/1069] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:49.041807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44ced7-f23b-4a52-bd01-53cb4e5312d7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:49.076471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:49.138379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44ced7-f23b-4a52-bd01-53cb4e5312d7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:49.171168 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b4981-cb76-4843-8c0f-510b63ac8cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:49.171766 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b4981-cb76-4843-8c0f-510b63ac8cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:49.172040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b4981-cb76-4843-8c0f-510b63ac8cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:49.172309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b4981-cb76-4843-8c0f-510b63ac8cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:49.172946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b4981-cb76-4843-8c0f-510b63ac8cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:49.173247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b4981-cb76-4843-8c0f-510b63ac8cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:49.174657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b4981-cb76-4843-8c0f-510b63ac8cf6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:49.186992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 542/1070] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:49.193034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1382f04-a46a-4e11-be84-786a61dc8ee4 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:49.288441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1382f04-a46a-4e11-be84-786a61dc8ee4 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:49.310101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:49.321288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d28413-bb2f-4433-8029-71e41921998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:49.321898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d28413-bb2f-4433-8029-71e41921998b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:49.322198 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d28413-bb2f-4433-8029-71e41921998b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:49.322469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d28413-bb2f-4433-8029-71e41921998b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:49.329485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d28413-bb2f-4433-8029-71e41921998b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:49.330915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d28413-bb2f-4433-8029-71e41921998b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-b3d28413-bb2f-4433-8029-71e41921998b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:49.341898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:49.343412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:49.344380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:49.344380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:49.344380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:49.344380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:49.344380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:49.344380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:49.344380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:49.344380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:49.344380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:25:49.344380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:49.349683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 529/1071] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 05:25:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:49.349799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b4981-cb76-4843-8c0f-510b63ac8cf6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:49.439493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:49.445525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b4981-cb76-4843-8c0f-510b63ac8cf6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:49.478867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70320a7-602c-460d-aca5-151bb6996882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:49.479473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70320a7-602c-460d-aca5-151bb6996882 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:49.479754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70320a7-602c-460d-aca5-151bb6996882 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:49.480028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70320a7-602c-460d-aca5-151bb6996882 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:49.480753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70320a7-602c-460d-aca5-151bb6996882 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:49.481062 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70320a7-602c-460d-aca5-151bb6996882 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:49.482520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70320a7-602c-460d-aca5-151bb6996882 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:49.499141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 543/1072] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 05:25:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:49.505081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d28413-bb2f-4433-8029-71e41921998b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:49.602549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d28413-bb2f-4433-8029-71e41921998b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:49.633992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4c4c0-d49b-4e9d-b005-f3ba7f3decc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:49.634633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4c4c0-d49b-4e9d-b005-f3ba7f3decc6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:49.634912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4c4c0-d49b-4e9d-b005-f3ba7f3decc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:49.635185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4c4c0-d49b-4e9d-b005-f3ba7f3decc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:49.645669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4c4c0-d49b-4e9d-b005-f3ba7f3decc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:49.647118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4c4c0-d49b-4e9d-b005-f3ba7f3decc6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-50e4c4c0-d49b-4e9d-b005-f3ba7f3decc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:49.661146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:49.662109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:49.663055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:49.663055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:49.663055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:49.663055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 530/1073] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:49.667965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70320a7-602c-460d-aca5-151bb6996882 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:49.764050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70320a7-602c-460d-aca5-151bb6996882 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:49.796916 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f780e2f-d937-49af-b3ee-c62737751fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:49.797518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f780e2f-d937-49af-b3ee-c62737751fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:49.797797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f780e2f-d937-49af-b3ee-c62737751fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:49.798073 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f780e2f-d937-49af-b3ee-c62737751fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:49.798737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f780e2f-d937-49af-b3ee-c62737751fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:49.799042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f780e2f-d937-49af-b3ee-c62737751fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:49.800403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f780e2f-d937-49af-b3ee-c62737751fb2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:49.811642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 544/1074] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:49.817241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4c4c0-d49b-4e9d-b005-f3ba7f3decc6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:49.851625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:49.917734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4c4c0-d49b-4e9d-b005-f3ba7f3decc6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:49.951507 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7dec3-ca7f-48ad-b856-600dbcbee9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:49.952680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7dec3-ca7f-48ad-b856-600dbcbee9bc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:49.952961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7dec3-ca7f-48ad-b856-600dbcbee9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:49.953232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7dec3-ca7f-48ad-b856-600dbcbee9bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:49.959757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7dec3-ca7f-48ad-b856-600dbcbee9bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:49.961180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7dec3-ca7f-48ad-b856-600dbcbee9bc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-afc7dec3-ca7f-48ad-b856-600dbcbee9bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:49.972201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:49.973092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:49.974369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:49.974369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:49.974369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:49.974369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:49.974369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:49.974369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:49.974369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:49.974369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:49.974369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 531/1075] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:49.979061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f780e2f-d937-49af-b3ee-c62737751fb2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:49.994954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:50.078077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f780e2f-d937-49af-b3ee-c62737751fb2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:50.111233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64954b-f65a-4abe-ae00-a43fbd365a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:50.111844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64954b-f65a-4abe-ae00-a43fbd365a8b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:50.112123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64954b-f65a-4abe-ae00-a43fbd365a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:50.112398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64954b-f65a-4abe-ae00-a43fbd365a8b None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:50.113086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64954b-f65a-4abe-ae00-a43fbd365a8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:50.113392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64954b-f65a-4abe-ae00-a43fbd365a8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:50.114856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64954b-f65a-4abe-ae00-a43fbd365a8b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:50.127109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 545/1076] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:50.133040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7dec3-ca7f-48ad-b856-600dbcbee9bc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:50.224315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:50.230901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7dec3-ca7f-48ad-b856-600dbcbee9bc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:50.263934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34c9c0-0def-4e52-95f0-5048b47bab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:50.264676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34c9c0-0def-4e52-95f0-5048b47bab52 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:50.264957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34c9c0-0def-4e52-95f0-5048b47bab52 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:50.265233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34c9c0-0def-4e52-95f0-5048b47bab52 None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/fba4f0473f4c4d3bb2d50b09f3de9837/roles/ad98bf0d9c9240e48f0243790aff940d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:50.267381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34c9c0-0def-4e52-95f0-5048b47bab52 None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=fba4f0473f4c4d3bb2d50b09f3de9837, role_id=ad98bf0d9c9240e48f0243790aff940d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:50.268801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34c9c0-0def-4e52-95f0-5048b47bab52 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:50.328566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 532/1077] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:25:50 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/fba4f0473f4c4d3bb2d50b09f3de9837/roles/ad98bf0d9c9240e48f0243790aff940d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:25:50.457109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307865e-5f0b-4dd9-a14a-9c10c97a2fef None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:50.457669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307865e-5f0b-4dd9-a14a-9c10c97a2fef None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:50.457669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307865e-5f0b-4dd9-a14a-9c10c97a2fef None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:50.459061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 546/1078] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:50.464797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1861983-0b01-4db5-9796-f01cb64db661 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:50.465084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1861983-0b01-4db5-9796-f01cb64db661 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:50.465359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1861983-0b01-4db5-9796-f01cb64db661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:50.484552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b1861983-0b01-4db5-9796-f01cb64db661 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:50.485785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b1861983-0b01-4db5-9796-f01cb64db661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:50.552893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 533/1079] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:50.815532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307865e-5f0b-4dd9-a14a-9c10c97a2fef None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:50.891567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:50.917305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307865e-5f0b-4dd9-a14a-9c10c97a2fef None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:50.949699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0011282-993b-40d3-a57d-8f53106ab3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:50.950352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0011282-993b-40d3-a57d-8f53106ab3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:50.950843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0011282-993b-40d3-a57d-8f53106ab3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:50.951301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0011282-993b-40d3-a57d-8f53106ab3c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:50.962967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0011282-993b-40d3-a57d-8f53106ab3c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-f0011282-993b-40d3-a57d-8f53106ab3c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:50.968452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:50.969633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:50.970528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 547/1080] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:50.975833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1861983-0b01-4db5-9796-f01cb64db661 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:51.057337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:51.072805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1861983-0b01-4db5-9796-f01cb64db661 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:51.106419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5f47-eba3-485f-bbb1-88b3e2110749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:51.107017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5f47-eba3-485f-bbb1-88b3e2110749 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:51.107294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5f47-eba3-485f-bbb1-88b3e2110749 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:51.107567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5f47-eba3-485f-bbb1-88b3e2110749 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:51.108279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a5f47-eba3-485f-bbb1-88b3e2110749 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:51.108579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a5f47-eba3-485f-bbb1-88b3e2110749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:51.109986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a5f47-eba3-485f-bbb1-88b3e2110749 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:51.118450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 534/1081] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:25:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:51.124236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0011282-993b-40d3-a57d-8f53106ab3c7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:51.224604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0011282-993b-40d3-a57d-8f53106ab3c7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:51.256883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb899efb-2149-4129-8c0d-deff0503e133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:51.257505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb899efb-2149-4129-8c0d-deff0503e133 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:51.257780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb899efb-2149-4129-8c0d-deff0503e133 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:51.258046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb899efb-2149-4129-8c0d-deff0503e133 None admin] PATH_INFO: `/v3/users/897f2b46ebe441949f1a01ccf9b9bee0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:51.259180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb899efb-2149-4129-8c0d-deff0503e133 None admin] RBAC: Authorizing `identity:get_user(user_id=897f2b46ebe441949f1a01ccf9b9bee0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:51.260591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb899efb-2149-4129-8c0d-deff0503e133 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:51.262176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 548/1082] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:25:51 2022] GET /identity/v3/users/897f2b46ebe441949f1a01ccf9b9bee0 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:51.268481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5f47-eba3-485f-bbb1-88b3e2110749 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:51.366898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5f47-eba3-485f-bbb1-88b3e2110749 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:51.399623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40480567-c8d8-49e5-be14-587e02b12a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:51.400246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40480567-c8d8-49e5-be14-587e02b12a54 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:51.400524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40480567-c8d8-49e5-be14-587e02b12a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:51.400796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40480567-c8d8-49e5-be14-587e02b12a54 None admin] PATH_INFO: `/v3/projects/1638d5fc33024482b7a64d56443f4c0c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:51.401896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40480567-c8d8-49e5-be14-587e02b12a54 None admin] RBAC: Authorizing `identity:get_project(project_id=1638d5fc33024482b7a64d56443f4c0c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:51.403332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40480567-c8d8-49e5-be14-587e02b12a54 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:51.404976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 535/1083] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:25:51 2022] GET /identity/v3/projects/1638d5fc33024482b7a64d56443f4c0c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:51.410853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb899efb-2149-4129-8c0d-deff0503e133 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:51.413709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:51.508293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb899efb-2149-4129-8c0d-deff0503e133 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:51.541346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a7b9f-9410-4e94-98a9-594993015f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:51.542009 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a7b9f-9410-4e94-98a9-594993015f3e None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:51.542328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a7b9f-9410-4e94-98a9-594993015f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:51.542603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a7b9f-9410-4e94-98a9-594993015f3e None admin] PATH_INFO: `/v3/projects/1638d5fc33024482b7a64d56443f4c0c/users/897f2b46ebe441949f1a01ccf9b9bee0/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:51.544488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a7b9f-9410-4e94-98a9-594993015f3e None admin] RBAC: Authorizing `identity:create_grant(project_id=1638d5fc33024482b7a64d56443f4c0c, user_id=897f2b46ebe441949f1a01ccf9b9bee0, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:51.545937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a7b9f-9410-4e94-98a9-594993015f3e None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:51.598357 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 549/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:25:51 2022] PUT /identity/v3/projects/1638d5fc33024482b7a64d56443f4c0c/users/897f2b46ebe441949f1a01ccf9b9bee0/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:25:52.445128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5282b0-6cb6-4588-ac28-977156ebae09 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:52.445681 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5282b0-6cb6-4588-ac28-977156ebae09 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:52.445797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5282b0-6cb6-4588-ac28-977156ebae09 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:52.447243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 536/1085] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:52.453772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefe9dd-3aa1-4f51-805d-c87b2ec85c3c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:52.454053 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefe9dd-3aa1-4f51-805d-c87b2ec85c3c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:52.454379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefe9dd-3aa1-4f51-805d-c87b2ec85c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:52.474003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ffefe9dd-3aa1-4f51-805d-c87b2ec85c3c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:52.475252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ffefe9dd-3aa1-4f51-805d-c87b2ec85c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:52.528528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:52.539156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 550/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:52.806960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5282b0-6cb6-4588-ac28-977156ebae09 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:52.823543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:52.908660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5282b0-6cb6-4588-ac28-977156ebae09 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:52.941375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12553347-84fb-4b32-8415-8e66d723eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:52.941997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553347-84fb-4b32-8415-8e66d723eeea None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:52.942319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553347-84fb-4b32-8415-8e66d723eeea None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:52.942715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553347-84fb-4b32-8415-8e66d723eeea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:52.953209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12553347-84fb-4b32-8415-8e66d723eeea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-12553347-84fb-4b32-8415-8e66d723eeea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:52.957970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:52.959113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:52.959987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 537/1087] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 05:25:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:52.965191 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefe9dd-3aa1-4f51-805d-c87b2ec85c3c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:53.063221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefe9dd-3aa1-4f51-805d-c87b2ec85c3c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:53.095808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b0e3d-6da4-4743-bc79-27f8df859bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:53.096401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b0e3d-6da4-4743-bc79-27f8df859bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:53.096674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b0e3d-6da4-4743-bc79-27f8df859bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:53.096940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b0e3d-6da4-4743-bc79-27f8df859bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:53.097588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b0e3d-6da4-4743-bc79-27f8df859bf7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:53.097887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b0e3d-6da4-4743-bc79-27f8df859bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:53.099266 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b0e3d-6da4-4743-bc79-27f8df859bf7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:53.106992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 551/1088] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 05:25:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:53.113405 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12553347-84fb-4b32-8415-8e66d723eeea None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:53.195055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:53.220216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12553347-84fb-4b32-8415-8e66d723eeea None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:53.253060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e498c-358d-49e4-be4b-77c8f038e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:53.253676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e498c-358d-49e4-be4b-77c8f038e68d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:53.253951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e498c-358d-49e4-be4b-77c8f038e68d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:53.254244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e498c-358d-49e4-be4b-77c8f038e68d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:53.263684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e498c-358d-49e4-be4b-77c8f038e68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:53.265134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e498c-358d-49e4-be4b-77c8f038e68d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-be0e498c-358d-49e4-be4b-77c8f038e68d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:53.279254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:53.280342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:53.281692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:53.281692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:53.281692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:53.281692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 538/1089] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:53.286231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b0e3d-6da4-4743-bc79-27f8df859bf7 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:53.340609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:53.385008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b0e3d-6da4-4743-bc79-27f8df859bf7 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:53.419384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9f31a-0373-4f98-9ee0-da8fbb2d665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:53.420058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9f31a-0373-4f98-9ee0-da8fbb2d665a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:53.420849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9f31a-0373-4f98-9ee0-da8fbb2d665a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:53.421125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9f31a-0373-4f98-9ee0-da8fbb2d665a None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:53.421946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9f31a-0373-4f98-9ee0-da8fbb2d665a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:53.422340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9f31a-0373-4f98-9ee0-da8fbb2d665a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:53.423777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9f31a-0373-4f98-9ee0-da8fbb2d665a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:53.435320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 552/1090] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:53.441534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e498c-358d-49e4-be4b-77c8f038e68d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:53.537998 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e498c-358d-49e4-be4b-77c8f038e68d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:53.562617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:53.572128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f390d-9f60-4b34-a2ef-d700220a1e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:53.576256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f390d-9f60-4b34-a2ef-d700220a1e85 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:53.576256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f390d-9f60-4b34-a2ef-d700220a1e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:53.576256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f390d-9f60-4b34-a2ef-d700220a1e85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:53.580698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f390d-9f60-4b34-a2ef-d700220a1e85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:53.582116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f390d-9f60-4b34-a2ef-d700220a1e85 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-3c0f390d-9f60-4b34-a2ef-d700220a1e85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:25:53.594425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:53.595739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:53.597232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 539/1091] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 05:25:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:53.601738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9f31a-0373-4f98-9ee0-da8fbb2d665a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:53.700372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9f31a-0373-4f98-9ee0-da8fbb2d665a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:53.711132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346be22-4cb9-4941-a5d1-30f9f81beac5 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:53.711430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346be22-4cb9-4941-a5d1-30f9f81beac5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:53.711710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346be22-4cb9-4941-a5d1-30f9f81beac5 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:53.713041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 540/1092] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:25:53.714958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:53.718546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b096f68-9daa-4a41-9052-b9a23e79c17c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:53.724781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b096f68-9daa-4a41-9052-b9a23e79c17c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:53.724781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b096f68-9daa-4a41-9052-b9a23e79c17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:53.736541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90bfac-76d1-40ce-ae98-1887a72a0baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:53.737147 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90bfac-76d1-40ce-ae98-1887a72a0baf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:53.737451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90bfac-76d1-40ce-ae98-1887a72a0baf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:53.737554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1b096f68-9daa-4a41-9052-b9a23e79c17c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:53.737704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90bfac-76d1-40ce-ae98-1887a72a0baf None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:53.738471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90bfac-76d1-40ce-ae98-1887a72a0baf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:53.738756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1b096f68-9daa-4a41-9052-b9a23e79c17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:53.738971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90bfac-76d1-40ce-ae98-1887a72a0baf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:53.740809 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90bfac-76d1-40ce-ae98-1887a72a0baf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:53.757892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 553/1093] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 05:25:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:53.763595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90bfac-76d1-40ce-ae98-1887a72a0baf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:53.804420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 541/1094] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:25:53.857476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90bfac-76d1-40ce-ae98-1887a72a0baf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:53.890480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69587259-812e-410d-bc01-45c1d12d8a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:53.891260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69587259-812e-410d-bc01-45c1d12d8a9f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:53.891373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69587259-812e-410d-bc01-45c1d12d8a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:53.891828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69587259-812e-410d-bc01-45c1d12d8a9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:53.951947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:53.977404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69587259-812e-410d-bc01-45c1d12d8a9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:53.978870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69587259-812e-410d-bc01-45c1d12d8a9f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:54.071286 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 554/1095] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 05:25:54.075247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b096f68-9daa-4a41-9052-b9a23e79c17c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:54.077449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69587259-812e-410d-bc01-45c1d12d8a9f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:54.142848 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:54.171718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69587259-812e-410d-bc01-45c1d12d8a9f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:54.171895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b096f68-9daa-4a41-9052-b9a23e79c17c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:54.188074 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:54.205360 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b267b-b5d9-425e-a5e2-e34ed97b75fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:54.205986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b267b-b5d9-425e-a5e2-e34ed97b75fe None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:54.206106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627de269-fda7-4a80-b6be-8f8e4a4c04de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:54.206300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b267b-b5d9-425e-a5e2-e34ed97b75fe None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:54.206575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b267b-b5d9-425e-a5e2-e34ed97b75fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:54.206668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627de269-fda7-4a80-b6be-8f8e4a4c04de None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:54.206926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627de269-fda7-4a80-b6be-8f8e4a4c04de None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:54.207191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627de269-fda7-4a80-b6be-8f8e4a4c04de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:54.216908 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b267b-b5d9-425e-a5e2-e34ed97b75fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:54.217756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627de269-fda7-4a80-b6be-8f8e4a4c04de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:25:54.218936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b267b-b5d9-425e-a5e2-e34ed97b75fe None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-627de269-fda7-4a80-b6be-8f8e4a4c04de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:25:54.222607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:54.226181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:25:54.227297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 542/1096] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:25:54.229583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627de269-fda7-4a80-b6be-8f8e4a4c04de None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-e92b267b-b5d9-425e-a5e2-e34ed97b75fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:25:54.236332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:54.237443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:25:54.238379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:25:54.238379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:25:54.238379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:54.238379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 555/1097] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:54.242907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b267b-b5d9-425e-a5e2-e34ed97b75fe None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:54.329976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627de269-fda7-4a80-b6be-8f8e4a4c04de None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:54.345371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b267b-b5d9-425e-a5e2-e34ed97b75fe None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:54.363045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa234ea-3ee7-4e91-b872-1541e83f699d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:54.363643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa234ea-3ee7-4e91-b872-1541e83f699d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:54.363924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa234ea-3ee7-4e91-b872-1541e83f699d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:54.364194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa234ea-3ee7-4e91-b872-1541e83f699d None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:54.364842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa234ea-3ee7-4e91-b872-1541e83f699d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:54.365140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa234ea-3ee7-4e91-b872-1541e83f699d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:54.366515 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa234ea-3ee7-4e91-b872-1541e83f699d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:54.367320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:54.374732 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 543/1098] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:54.378235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108fb45-c3e5-4ad1-9663-c872bdb04bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:54.380057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108fb45-c3e5-4ad1-9663-c872bdb04bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:54.380057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108fb45-c3e5-4ad1-9663-c872bdb04bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:54.380057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108fb45-c3e5-4ad1-9663-c872bdb04bc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:54.380432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108fb45-c3e5-4ad1-9663-c872bdb04bc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:54.380650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa234ea-3ee7-4e91-b872-1541e83f699d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:54.380746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108fb45-c3e5-4ad1-9663-c872bdb04bc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:54.382071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108fb45-c3e5-4ad1-9663-c872bdb04bc8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:54.395387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 556/1099] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:54.400513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108fb45-c3e5-4ad1-9663-c872bdb04bc8 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:54.419068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:54.477544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa234ea-3ee7-4e91-b872-1541e83f699d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:54.497098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108fb45-c3e5-4ad1-9663-c872bdb04bc8 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:54.511445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705218e-ad95-4e8b-8bda-15c1c7d70c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:54.512164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705218e-ad95-4e8b-8bda-15c1c7d70c47 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:54.512507 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705218e-ad95-4e8b-8bda-15c1c7d70c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:54.512849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705218e-ad95-4e8b-8bda-15c1c7d70c47 None admin] PATH_INFO: `/v3/users/897f2b46ebe441949f1a01ccf9b9bee0` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:54.514563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705218e-ad95-4e8b-8bda-15c1c7d70c47 None admin] RBAC: Authorizing `identity:get_user(user_id=897f2b46ebe441949f1a01ccf9b9bee0)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:54.515970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705218e-ad95-4e8b-8bda-15c1c7d70c47 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:54.517543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 544/1100] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/users/897f2b46ebe441949f1a01ccf9b9bee0 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:54.523332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705218e-ad95-4e8b-8bda-15c1c7d70c47 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:54.530535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938533d-c9f7-4065-9593-8b14d4466e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:54.531158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938533d-c9f7-4065-9593-8b14d4466e7a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:54.531431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938533d-c9f7-4065-9593-8b14d4466e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:54.531696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938533d-c9f7-4065-9593-8b14d4466e7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:54.538152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938533d-c9f7-4065-9593-8b14d4466e7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:54.539569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938533d-c9f7-4065-9593-8b14d4466e7a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8938533d-c9f7-4065-9593-8b14d4466e7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:25:54.550336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:25:54.551350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:25:54.557476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:25:54.557476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:25:54.557476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:25:54.557476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:25:54.557476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:25:54.557476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:25:54.557476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:25:54.557476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:25:54.557476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 557/1101] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:25:54.557988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938533d-c9f7-4065-9593-8b14d4466e7a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:54.602197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:54.624088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705218e-ad95-4e8b-8bda-15c1c7d70c47 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:54.631424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:54.657660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7385498-9e86-4e62-9a3c-d2c93fe6ddad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:54.658242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7385498-9e86-4e62-9a3c-d2c93fe6ddad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:54.658520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7385498-9e86-4e62-9a3c-d2c93fe6ddad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:54.658788 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7385498-9e86-4e62-9a3c-d2c93fe6ddad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:54.660028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938533d-c9f7-4065-9593-8b14d4466e7a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:54.693640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4a978-4f45-4b6d-9632-68afa165cd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:54.694532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4a978-4f45-4b6d-9632-68afa165cd3a None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:54.694806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4a978-4f45-4b6d-9632-68afa165cd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:54.695072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4a978-4f45-4b6d-9632-68afa165cd3a None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:54.695742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4a978-4f45-4b6d-9632-68afa165cd3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:54.696044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4a978-4f45-4b6d-9632-68afa165cd3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:54.697459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4a978-4f45-4b6d-9632-68afa165cd3a None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:54.709127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 558/1102] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:54.715193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4a978-4f45-4b6d-9632-68afa165cd3a None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:54.726598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7385498-9e86-4e62-9a3c-d2c93fe6ddad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:54.728042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7385498-9e86-4e62-9a3c-d2c93fe6ddad None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:54.812303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4a978-4f45-4b6d-9632-68afa165cd3a None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:54.814587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:54.825124 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 545/1103] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 05:25:54.831118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7385498-9e86-4e62-9a3c-d2c93fe6ddad None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:54.838516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:54.846808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0ca3-42db-41ed-adcc-15b7147f8996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:54.847443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0ca3-42db-41ed-adcc-15b7147f8996 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:54.847725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0ca3-42db-41ed-adcc-15b7147f8996 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:54.848000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb0ca3-42db-41ed-adcc-15b7147f8996 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:54.848724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb0ca3-42db-41ed-adcc-15b7147f8996 None admin] RBAC: Adding query filter params (role.id=ad98bf0d9c9240e48f0243790aff940d, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=fba4f0473f4c4d3bb2d50b09f3de9837) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:54.849045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb0ca3-42db-41ed-adcc-15b7147f8996 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:54.850521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb0ca3-42db-41ed-adcc-15b7147f8996 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:54.866086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 559/1104] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/role_assignments?user.id=fba4f0473f4c4d3bb2d50b09f3de9837&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=ad98bf0d9c9240e48f0243790aff940d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:54.938506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7385498-9e86-4e62-9a3c-d2c93fe6ddad None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:54.973005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f063ca-4da6-45a7-a9dc-dc6fe50f3176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:54.973643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f063ca-4da6-45a7-a9dc-dc6fe50f3176 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:54.973917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f063ca-4da6-45a7-a9dc-dc6fe50f3176 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:54.974209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f063ca-4da6-45a7-a9dc-dc6fe50f3176 None admin] PATH_INFO: `/v3/projects/1638d5fc33024482b7a64d56443f4c0c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:54.975336 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f063ca-4da6-45a7-a9dc-dc6fe50f3176 None admin] RBAC: Authorizing `identity:get_project(project_id=1638d5fc33024482b7a64d56443f4c0c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:54.976773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f063ca-4da6-45a7-a9dc-dc6fe50f3176 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:54.978516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 546/1105] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/projects/1638d5fc33024482b7a64d56443f4c0c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:25:54.985456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0ca3-42db-41ed-adcc-15b7147f8996 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:55.086619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb0ca3-42db-41ed-adcc-15b7147f8996 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:55.120920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b45fe-0509-4444-8c73-adfa7a46efc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:55.121527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b45fe-0509-4444-8c73-adfa7a46efc3 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:55.121804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b45fe-0509-4444-8c73-adfa7a46efc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:55.122078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b45fe-0509-4444-8c73-adfa7a46efc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:55.122834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b45fe-0509-4444-8c73-adfa7a46efc3 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=1638d5fc33024482b7a64d56443f4c0c, user.id=897f2b46ebe441949f1a01ccf9b9bee0) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:25:55.123171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b45fe-0509-4444-8c73-adfa7a46efc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:55.124597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b45fe-0509-4444-8c73-adfa7a46efc3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:55.130412 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 560/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:25:54 2022] GET /identity/v3/role_assignments?user.id=897f2b46ebe441949f1a01ccf9b9bee0&scope.project.id=1638d5fc33024482b7a64d56443f4c0c&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:57.284301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba20f3-0881-4cac-bd82-62695d71adf0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:57.284802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba20f3-0881-4cac-bd82-62695d71adf0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:57.284802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba20f3-0881-4cac-bd82-62695d71adf0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:57.286100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 547/1107] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:25:57.291209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8652c-88cc-470c-913c-a6007b1c47b3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:57.291635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8652c-88cc-470c-913c-a6007b1c47b3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:57.291753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8652c-88cc-470c-913c-a6007b1c47b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:57.309877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-75a8652c-88cc-470c-913c-a6007b1c47b3 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:25:57.311082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-75a8652c-88cc-470c-913c-a6007b1c47b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:25:57.335309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:57.375433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 561/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:25:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:25:57.640743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba20f3-0881-4cac-bd82-62695d71adf0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:57.715039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:25:57.739002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba20f3-0881-4cac-bd82-62695d71adf0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:57.770807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bf92c-2707-44d6-86a1-68acb7e6a395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:57.771437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bf92c-2707-44d6-86a1-68acb7e6a395 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:57.771716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bf92c-2707-44d6-86a1-68acb7e6a395 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:57.771990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bf92c-2707-44d6-86a1-68acb7e6a395 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:57.773105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bf92c-2707-44d6-86a1-68acb7e6a395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:57.774542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bf92c-2707-44d6-86a1-68acb7e6a395 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:57.776090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 548/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:25:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:25:57.782407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8652c-88cc-470c-913c-a6007b1c47b3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:25:57.874481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8652c-88cc-470c-913c-a6007b1c47b3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:25:57.907875 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28bb7e-761b-4048-8c2b-1d1ea471cf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:25:57.908418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28bb7e-761b-4048-8c2b-1d1ea471cf36 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:25:57.908693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28bb7e-761b-4048-8c2b-1d1ea471cf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:25:57.908962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28bb7e-761b-4048-8c2b-1d1ea471cf36 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:25:57.909612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28bb7e-761b-4048-8c2b-1d1ea471cf36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:25:57.911066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28bb7e-761b-4048-8c2b-1d1ea471cf36 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:25:58.022331 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 562/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:25:57 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:26:00.196557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b162521-e596-4b05-8969-4bf3f9464abe None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:00.197070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b162521-e596-4b05-8969-4bf3f9464abe None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:00.197070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b162521-e596-4b05-8969-4bf3f9464abe None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:00.198373 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 549/1111] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:00.203526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed4f1-64f1-427f-9000-d6c40cb78403 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:00.203887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed4f1-64f1-427f-9000-d6c40cb78403 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:00.204076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed4f1-64f1-427f-9000-d6c40cb78403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:00.221701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e30ed4f1-64f1-427f-9000-d6c40cb78403 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:00.222907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e30ed4f1-64f1-427f-9000-d6c40cb78403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:00.235931 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:00.288082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 563/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:00.560263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b162521-e596-4b05-8969-4bf3f9464abe None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:00.657687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b162521-e596-4b05-8969-4bf3f9464abe None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:00.689237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffdcbc-a28c-437d-aa50-e03650dc9f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:00.689843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffdcbc-a28c-437d-aa50-e03650dc9f90 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:00.690118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffdcbc-a28c-437d-aa50-e03650dc9f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:00.690417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffdcbc-a28c-437d-aa50-e03650dc9f90 None admin] PATH_INFO: `/v3/roles/cae161e9c1b64656924448dd784ed4d1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:00.691777 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffdcbc-a28c-437d-aa50-e03650dc9f90 None admin] RBAC: Authorizing `identity:get_role(role_id=cae161e9c1b64656924448dd784ed4d1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:00.693208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffdcbc-a28c-437d-aa50-e03650dc9f90 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:00.694773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 550/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:00 2022] GET /identity/v3/roles/cae161e9c1b64656924448dd784ed4d1 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:00.700743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ed4f1-64f1-427f-9000-d6c40cb78403 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:00.796096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ed4f1-64f1-427f-9000-d6c40cb78403 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:00.828723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ad0fa-5882-4489-8aa2-130f004c8074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:00.829282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ad0fa-5882-4489-8aa2-130f004c8074 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:00.829560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ad0fa-5882-4489-8aa2-130f004c8074 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:00.829832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ad0fa-5882-4489-8aa2-130f004c8074 None admin] PATH_INFO: `/v3/users/ec35e6ffa7654304b2076a7151d54f39` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:00.842808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ad0fa-5882-4489-8aa2-130f004c8074 None admin] RBAC: Authorizing `identity:get_user(user_id=ec35e6ffa7654304b2076a7151d54f39)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:00.844230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ad0fa-5882-4489-8aa2-130f004c8074 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:00.845798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 564/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:00 2022] GET /identity/v3/users/ec35e6ffa7654304b2076a7151d54f39 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:00.851524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffdcbc-a28c-437d-aa50-e03650dc9f90 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:00.863792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:00.948334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffdcbc-a28c-437d-aa50-e03650dc9f90 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:00.980770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8492532-874a-41ab-b2dc-0380bd189800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:00.981365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8492532-874a-41ab-b2dc-0380bd189800 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:00.981642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8492532-874a-41ab-b2dc-0380bd189800 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:00.981915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8492532-874a-41ab-b2dc-0380bd189800 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:01.045854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8492532-874a-41ab-b2dc-0380bd189800 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:01.047345 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8492532-874a-41ab-b2dc-0380bd189800 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:01.057866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:01.141798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 551/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:26:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:26:01.147722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ad0fa-5882-4489-8aa2-130f004c8074 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:01.156434 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:01.242763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ad0fa-5882-4489-8aa2-130f004c8074 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:01.275296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e47c3-1a0e-4b08-bb84-ae570b86e34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:01.275913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e47c3-1a0e-4b08-bb84-ae570b86e34b None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:01.276187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e47c3-1a0e-4b08-bb84-ae570b86e34b None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:01.276463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e47c3-1a0e-4b08-bb84-ae570b86e34b None admin] PATH_INFO: `/v3/projects/1638d5fc33024482b7a64d56443f4c0c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:01.277567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e47c3-1a0e-4b08-bb84-ae570b86e34b None admin] RBAC: Authorizing `identity:get_project(project_id=1638d5fc33024482b7a64d56443f4c0c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:01.279144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e47c3-1a0e-4b08-bb84-ae570b86e34b None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:01.280742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 565/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:26:01 2022] GET /identity/v3/projects/1638d5fc33024482b7a64d56443f4c0c => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:01.286709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8492532-874a-41ab-b2dc-0380bd189800 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:01.381801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8492532-874a-41ab-b2dc-0380bd189800 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:01.391992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:01.413801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d5a5c-fd39-427a-9c6e-6618c79a2fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:01.414428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d5a5c-fd39-427a-9c6e-6618c79a2fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:01.414710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d5a5c-fd39-427a-9c6e-6618c79a2fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:01.414983 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d5a5c-fd39-427a-9c6e-6618c79a2fc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:01.415699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d5a5c-fd39-427a-9c6e-6618c79a2fc4 None admin] RBAC: Adding query filter params (role.id=cae161e9c1b64656924448dd784ed4d1, scope.project.id=1638d5fc33024482b7a64d56443f4c0c, user.id=ec35e6ffa7654304b2076a7151d54f39) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:01.416006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d5a5c-fd39-427a-9c6e-6618c79a2fc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:01.417413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d5a5c-fd39-427a-9c6e-6618c79a2fc4 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:01.423043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 552/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:26:01 2022] GET /identity/v3/role_assignments?user.id=ec35e6ffa7654304b2076a7151d54f39&scope.project.id=1638d5fc33024482b7a64d56443f4c0c&role.id=cae161e9c1b64656924448dd784ed4d1 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:03.521889 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60df66-f116-4df6-bfad-f107ef5d7ae4 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:03.522473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60df66-f116-4df6-bfad-f107ef5d7ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:03.522473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60df66-f116-4df6-bfad-f107ef5d7ae4 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:03.523798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 566/1118] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:03.529001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1db64f-677d-4611-992c-013e2b42b1e6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:03.529392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1db64f-677d-4611-992c-013e2b42b1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:03.529564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1db64f-677d-4611-992c-013e2b42b1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:03.547817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8e1db64f-677d-4611-992c-013e2b42b1e6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:03.549011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8e1db64f-677d-4611-992c-013e2b42b1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:03.613588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 553/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:03.889269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60df66-f116-4df6-bfad-f107ef5d7ae4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:03.962840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:03.987275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60df66-f116-4df6-bfad-f107ef5d7ae4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:04.020811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005f179-6a5e-49b0-b4b1-9a4393a93514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:04.021421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005f179-6a5e-49b0-b4b1-9a4393a93514 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:04.021697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005f179-6a5e-49b0-b4b1-9a4393a93514 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:04.021964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005f179-6a5e-49b0-b4b1-9a4393a93514 None admin] PATH_INFO: `/v3/roles/cae161e9c1b64656924448dd784ed4d1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:04.023362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005f179-6a5e-49b0-b4b1-9a4393a93514 None admin] RBAC: Authorizing `identity:get_role(role_id=cae161e9c1b64656924448dd784ed4d1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:04.024712 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005f179-6a5e-49b0-b4b1-9a4393a93514 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:04.026003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 567/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:03 2022] GET /identity/v3/roles/cae161e9c1b64656924448dd784ed4d1 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:04.032262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1db64f-677d-4611-992c-013e2b42b1e6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:04.126446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1db64f-677d-4611-992c-013e2b42b1e6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:04.128679 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:04.159156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14554d5-e831-48d1-9ade-6fd3e3bca2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:04.159712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14554d5-e831-48d1-9ade-6fd3e3bca2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:04.159985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14554d5-e831-48d1-9ade-6fd3e3bca2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:04.160249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14554d5-e831-48d1-9ade-6fd3e3bca2d8 None admin] PATH_INFO: `/v3/users/ec35e6ffa7654304b2076a7151d54f39` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:04.161382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14554d5-e831-48d1-9ade-6fd3e3bca2d8 None admin] RBAC: Authorizing `identity:get_user(user_id=ec35e6ffa7654304b2076a7151d54f39)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:04.162810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14554d5-e831-48d1-9ade-6fd3e3bca2d8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:04.164347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 554/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:04 2022] GET /identity/v3/users/ec35e6ffa7654304b2076a7151d54f39 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:04.170546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005f179-6a5e-49b0-b4b1-9a4393a93514 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:04.265119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005f179-6a5e-49b0-b4b1-9a4393a93514 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:04.298549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0a43a-75c4-4b08-922b-cd951c8713ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:04.299193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0a43a-75c4-4b08-922b-cd951c8713ed None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:04.299473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0a43a-75c4-4b08-922b-cd951c8713ed None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:04.299746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0a43a-75c4-4b08-922b-cd951c8713ed None admin] PATH_INFO: `/v3/projects/1638d5fc33024482b7a64d56443f4c0c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:04.300862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0a43a-75c4-4b08-922b-cd951c8713ed None admin] RBAC: Authorizing `identity:get_project(project_id=1638d5fc33024482b7a64d56443f4c0c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:04.302306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0a43a-75c4-4b08-922b-cd951c8713ed None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:04.303917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 568/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:26:04 2022] GET /identity/v3/projects/1638d5fc33024482b7a64d56443f4c0c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:04.309954 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14554d5-e831-48d1-9ade-6fd3e3bca2d8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:04.405173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14554d5-e831-48d1-9ade-6fd3e3bca2d8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:04.436895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444d2b4-df23-4a19-a0ce-6ec664ddfd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:04.437474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444d2b4-df23-4a19-a0ce-6ec664ddfd9a None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:04.437749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444d2b4-df23-4a19-a0ce-6ec664ddfd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:04.438019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444d2b4-df23-4a19-a0ce-6ec664ddfd9a None admin] PATH_INFO: `/v3/projects/1638d5fc33024482b7a64d56443f4c0c/users/ec35e6ffa7654304b2076a7151d54f39/roles/cae161e9c1b64656924448dd784ed4d1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:04.440136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444d2b4-df23-4a19-a0ce-6ec664ddfd9a None admin] RBAC: Authorizing `identity:create_grant(project_id=1638d5fc33024482b7a64d56443f4c0c, user_id=ec35e6ffa7654304b2076a7151d54f39, role_id=cae161e9c1b64656924448dd784ed4d1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:04.441546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444d2b4-df23-4a19-a0ce-6ec664ddfd9a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:04.503729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 555/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:26:04 2022] PUT /identity/v3/projects/1638d5fc33024482b7a64d56443f4c0c/users/ec35e6ffa7654304b2076a7151d54f39/roles/cae161e9c1b64656924448dd784ed4d1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:26:06.614465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949707e-dd76-4fd5-9feb-7875df438684 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:06.615009 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949707e-dd76-4fd5-9feb-7875df438684 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:06.615009 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949707e-dd76-4fd5-9feb-7875df438684 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:06.616345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 569/1124] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:06.621499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce10e7-49d3-413b-b16e-884f18b17741 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:06.621916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce10e7-49d3-413b-b16e-884f18b17741 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:06.622029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce10e7-49d3-413b-b16e-884f18b17741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:06.640197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-20ce10e7-49d3-413b-b16e-884f18b17741 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:06.641374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20ce10e7-49d3-413b-b16e-884f18b17741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:06.665719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:06.704891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 556/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:06.974826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949707e-dd76-4fd5-9feb-7875df438684 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:06.987525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:07.074144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949707e-dd76-4fd5-9feb-7875df438684 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:07.106681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bed6a9-4576-431d-a04b-b8136ad664cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:07.107915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bed6a9-4576-431d-a04b-b8136ad664cf None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:07.108193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bed6a9-4576-431d-a04b-b8136ad664cf None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:07.108467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bed6a9-4576-431d-a04b-b8136ad664cf None admin] PATH_INFO: `/v3/roles/cae161e9c1b64656924448dd784ed4d1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:07.109849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bed6a9-4576-431d-a04b-b8136ad664cf None admin] RBAC: Authorizing `identity:get_role(role_id=cae161e9c1b64656924448dd784ed4d1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:07.111249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bed6a9-4576-431d-a04b-b8136ad664cf None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:07.112543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 570/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:06 2022] GET /identity/v3/roles/cae161e9c1b64656924448dd784ed4d1 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:07.118222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce10e7-49d3-413b-b16e-884f18b17741 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:07.211463 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce10e7-49d3-413b-b16e-884f18b17741 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:07.244109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ed855-ef13-4007-813e-6ee965e9e009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:07.244719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ed855-ef13-4007-813e-6ee965e9e009 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:07.244998 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ed855-ef13-4007-813e-6ee965e9e009 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:07.245268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ed855-ef13-4007-813e-6ee965e9e009 None admin] PATH_INFO: `/v3/users/ec35e6ffa7654304b2076a7151d54f39` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:07.246401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ed855-ef13-4007-813e-6ee965e9e009 None admin] RBAC: Authorizing `identity:get_user(user_id=ec35e6ffa7654304b2076a7151d54f39)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:07.247838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ed855-ef13-4007-813e-6ee965e9e009 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:07.249365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 557/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:07 2022] GET /identity/v3/users/ec35e6ffa7654304b2076a7151d54f39 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:07.254822 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bed6a9-4576-431d-a04b-b8136ad664cf None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:07.330092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:07.350948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bed6a9-4576-431d-a04b-b8136ad664cf None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:07.383624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0978c2-f58d-4139-bba1-6c15e0b2b486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:07.384212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0978c2-f58d-4139-bba1-6c15e0b2b486 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:07.384495 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0978c2-f58d-4139-bba1-6c15e0b2b486 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:07.384766 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0978c2-f58d-4139-bba1-6c15e0b2b486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:07.449242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0978c2-f58d-4139-bba1-6c15e0b2b486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:07.451226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0978c2-f58d-4139-bba1-6c15e0b2b486 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:07.524144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:07.545292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 571/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:26:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:26:07.551249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ed855-ef13-4007-813e-6ee965e9e009 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:07.581425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:07.651854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ed855-ef13-4007-813e-6ee965e9e009 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:07.685219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b368e62-708c-4db0-b1c1-670db53b6f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:07.685835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b368e62-708c-4db0-b1c1-670db53b6f1d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:07.686112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b368e62-708c-4db0-b1c1-670db53b6f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:07.686421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b368e62-708c-4db0-b1c1-670db53b6f1d None admin] PATH_INFO: `/v3/projects/1638d5fc33024482b7a64d56443f4c0c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:07.687526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b368e62-708c-4db0-b1c1-670db53b6f1d None admin] RBAC: Authorizing `identity:get_project(project_id=1638d5fc33024482b7a64d56443f4c0c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:07.688942 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b368e62-708c-4db0-b1c1-670db53b6f1d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:07.690571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 558/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:26:07 2022] GET /identity/v3/projects/1638d5fc33024482b7a64d56443f4c0c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:07.696968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0978c2-f58d-4139-bba1-6c15e0b2b486 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:07.791392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0978c2-f58d-4139-bba1-6c15e0b2b486 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:07.823676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e189402-9df6-4007-a6ef-047f2cbf80d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:07.824264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e189402-9df6-4007-a6ef-047f2cbf80d5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:07.824538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e189402-9df6-4007-a6ef-047f2cbf80d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:07.824807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e189402-9df6-4007-a6ef-047f2cbf80d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:07.825509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e189402-9df6-4007-a6ef-047f2cbf80d5 None admin] RBAC: Adding query filter params (role.id=cae161e9c1b64656924448dd784ed4d1, scope.project.id=1638d5fc33024482b7a64d56443f4c0c, user.id=ec35e6ffa7654304b2076a7151d54f39) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:07.826436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e189402-9df6-4007-a6ef-047f2cbf80d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:07.827832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e189402-9df6-4007-a6ef-047f2cbf80d5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:07.833297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 572/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:26:07 2022] GET /identity/v3/role_assignments?user.id=ec35e6ffa7654304b2076a7151d54f39&scope.project.id=1638d5fc33024482b7a64d56443f4c0c&role.id=cae161e9c1b64656924448dd784ed4d1 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:09.965122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6d554-d1b7-4b1e-96f9-c95a0a0d2edf None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:09.965644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6d554-d1b7-4b1e-96f9-c95a0a0d2edf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:09.965644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6d554-d1b7-4b1e-96f9-c95a0a0d2edf None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:09.967045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 559/1131] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:09.972104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede02310-a41c-435e-9233-e844b61f327f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:09.972519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede02310-a41c-435e-9233-e844b61f327f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:09.972651 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede02310-a41c-435e-9233-e844b61f327f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:09.991271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ede02310-a41c-435e-9233-e844b61f327f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:09.992437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ede02310-a41c-435e-9233-e844b61f327f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:10.025938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:10.056043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 573/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:10.323298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6d554-d1b7-4b1e-96f9-c95a0a0d2edf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:10.416433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:10.421487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6d554-d1b7-4b1e-96f9-c95a0a0d2edf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:10.454078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302f184-85dd-40b9-a2ac-c0a3bb7630ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:10.454723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302f184-85dd-40b9-a2ac-c0a3bb7630ad None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:10.455005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302f184-85dd-40b9-a2ac-c0a3bb7630ad None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:10.455280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302f184-85dd-40b9-a2ac-c0a3bb7630ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:10.456374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302f184-85dd-40b9-a2ac-c0a3bb7630ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:10.457781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302f184-85dd-40b9-a2ac-c0a3bb7630ad None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:10.459360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 560/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:26:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:10.465000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede02310-a41c-435e-9233-e844b61f327f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:10.559162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede02310-a41c-435e-9233-e844b61f327f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:10.591843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bed1c-48bc-4934-9dd8-7651744f023f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:10.592381 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bed1c-48bc-4934-9dd8-7651744f023f None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:10.592658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bed1c-48bc-4934-9dd8-7651744f023f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:10.592929 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bed1c-48bc-4934-9dd8-7651744f023f None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:10.593573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bed1c-48bc-4934-9dd8-7651744f023f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:10.595070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bed1c-48bc-4934-9dd8-7651744f023f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:10.673192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 574/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:26:10 2022] POST /identity/v3/projects => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:26:12.814454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb324-bc63-42b1-b342-01b90c609838 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:12.815003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb324-bc63-42b1-b342-01b90c609838 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:12.815003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcb324-bc63-42b1-b342-01b90c609838 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:12.816331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 561/1135] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:12.821763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a82e7-019a-4533-a4ea-43adc9942a5e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:12.822213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a82e7-019a-4533-a4ea-43adc9942a5e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:12.822345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a82e7-019a-4533-a4ea-43adc9942a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:12.840543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bd0a82e7-019a-4533-a4ea-43adc9942a5e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:12.842386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bd0a82e7-019a-4533-a4ea-43adc9942a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:12.858908 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:12.906904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 575/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:26:13.173831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcb324-bc63-42b1-b342-01b90c609838 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:13.271549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcb324-bc63-42b1-b342-01b90c609838 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:13.303006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9020934-dbfe-4885-a365-91e70ccae890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:13.303613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9020934-dbfe-4885-a365-91e70ccae890 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:13.303890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9020934-dbfe-4885-a365-91e70ccae890 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:13.304157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9020934-dbfe-4885-a365-91e70ccae890 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:13.305244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9020934-dbfe-4885-a365-91e70ccae890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:13.306675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9020934-dbfe-4885-a365-91e70ccae890 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:13.308200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 562/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:26:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:13.314345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a82e7-019a-4533-a4ea-43adc9942a5e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:13.408937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a82e7-019a-4533-a4ea-43adc9942a5e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:13.441857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f8118-dbc9-41a1-b46c-de6adeafd6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:13.442423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f8118-dbc9-41a1-b46c-de6adeafd6b4 None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:13.442703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f8118-dbc9-41a1-b46c-de6adeafd6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:13.442975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f8118-dbc9-41a1-b46c-de6adeafd6b4 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:13.443612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f8118-dbc9-41a1-b46c-de6adeafd6b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:13.445296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f8118-dbc9-41a1-b46c-de6adeafd6b4 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:13.550051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 576/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:26:13 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:26:15.704685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24b7a0-85d7-469c-b955-cb9c9b24602f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:15.705215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24b7a0-85d7-469c-b955-cb9c9b24602f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:15.705215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24b7a0-85d7-469c-b955-cb9c9b24602f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:15.706606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 563/1139] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:15.711898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46244-74ab-45a2-98af-b4c0f3b50284 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:15.712289 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46244-74ab-45a2-98af-b4c0f3b50284 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:15.712430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec46244-74ab-45a2-98af-b4c0f3b50284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:15.730685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cec46244-74ab-45a2-98af-b4c0f3b50284 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:15.731912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cec46244-74ab-45a2-98af-b4c0f3b50284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:15.734567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:15.795788 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 577/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:16.062547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24b7a0-85d7-469c-b955-cb9c9b24602f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:16.097743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:16.162068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24b7a0-85d7-469c-b955-cb9c9b24602f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:16.195219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea108c7-b50b-4589-aa4f-673b90c8e98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:16.195834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea108c7-b50b-4589-aa4f-673b90c8e98f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:16.196112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea108c7-b50b-4589-aa4f-673b90c8e98f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:16.196382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea108c7-b50b-4589-aa4f-673b90c8e98f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:16.206676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea108c7-b50b-4589-aa4f-673b90c8e98f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5ea108c7-b50b-4589-aa4f-673b90c8e98f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:26:16.211434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:26:16.212498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:26:16.213517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 564/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:26:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:26:16.218458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec46244-74ab-45a2-98af-b4c0f3b50284 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:16.312168 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec46244-74ab-45a2-98af-b4c0f3b50284 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:16.345194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5f3c1-d23c-42f3-a3e3-992b5cf81276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:16.345797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f3c1-d23c-42f3-a3e3-992b5cf81276 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:16.346073 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f3c1-d23c-42f3-a3e3-992b5cf81276 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:16.346375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f3c1-d23c-42f3-a3e3-992b5cf81276 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:16.347033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5f3c1-d23c-42f3-a3e3-992b5cf81276 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:16.347336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5f3c1-d23c-42f3-a3e3-992b5cf81276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:16.348687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5f3c1-d23c-42f3-a3e3-992b5cf81276 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:16.356391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 578/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:26:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:16.362841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea108c7-b50b-4589-aa4f-673b90c8e98f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:16.451724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:16.457989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea108c7-b50b-4589-aa4f-673b90c8e98f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:16.489887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f4322-e5cb-4a10-95c5-9bf72b173eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:16.490562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f4322-e5cb-4a10-95c5-9bf72b173eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:16.490837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f4322-e5cb-4a10-95c5-9bf72b173eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:16.491105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f4322-e5cb-4a10-95c5-9bf72b173eb6 None admin] PATH_INFO: `/v3/users/af2f5c83eeed4c998cf29f28dec5a1a0` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:16.503956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f4322-e5cb-4a10-95c5-9bf72b173eb6 None admin] RBAC: Authorizing `identity:get_user(user_id=af2f5c83eeed4c998cf29f28dec5a1a0)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:16.505365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f4322-e5cb-4a10-95c5-9bf72b173eb6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:16.506946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 565/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:16 2022] GET /identity/v3/users/af2f5c83eeed4c998cf29f28dec5a1a0 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:16.512777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5f3c1-d23c-42f3-a3e3-992b5cf81276 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:16.515610 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:16.610223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5f3c1-d23c-42f3-a3e3-992b5cf81276 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:16.642299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f438d-e641-426f-a3cf-2b615c11883c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:16.642899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f438d-e641-426f-a3cf-2b615c11883c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:16.643172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f438d-e641-426f-a3cf-2b615c11883c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:16.643441 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f438d-e641-426f-a3cf-2b615c11883c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:16.708789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f438d-e641-426f-a3cf-2b615c11883c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:16.710254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f438d-e641-426f-a3cf-2b615c11883c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:16.711974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:16.804816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 579/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:26:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:26:16.810751 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f4322-e5cb-4a10-95c5-9bf72b173eb6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:16.907735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f4322-e5cb-4a10-95c5-9bf72b173eb6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:16.941218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6200bd8-0550-4872-a3d5-356f5500290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:16.941834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6200bd8-0550-4872-a3d5-356f5500290e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:16.942114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6200bd8-0550-4872-a3d5-356f5500290e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:16.942420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6200bd8-0550-4872-a3d5-356f5500290e None admin] PATH_INFO: `/v3/projects/6ed26558a9c04df09e5b170c11abae62` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:16.943524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6200bd8-0550-4872-a3d5-356f5500290e None admin] RBAC: Authorizing `identity:get_project(project_id=6ed26558a9c04df09e5b170c11abae62)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:16.945186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6200bd8-0550-4872-a3d5-356f5500290e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:16.946845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 566/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:26:16 2022] GET /identity/v3/projects/6ed26558a9c04df09e5b170c11abae62 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:16.952792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f438d-e641-426f-a3cf-2b615c11883c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:17.047865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f438d-e641-426f-a3cf-2b615c11883c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:17.049337 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:17.080032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82788a-0008-4ffb-8d34-18c250457881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:17.080636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82788a-0008-4ffb-8d34-18c250457881 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:17.080911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82788a-0008-4ffb-8d34-18c250457881 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:17.081184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82788a-0008-4ffb-8d34-18c250457881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:17.081899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82788a-0008-4ffb-8d34-18c250457881 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=6ed26558a9c04df09e5b170c11abae62, user.id=af2f5c83eeed4c998cf29f28dec5a1a0) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:17.082233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82788a-0008-4ffb-8d34-18c250457881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:17.083968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82788a-0008-4ffb-8d34-18c250457881 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:17.089477 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 580/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:26:16 2022] GET /identity/v3/role_assignments?user.id=af2f5c83eeed4c998cf29f28dec5a1a0&scope.project.id=6ed26558a9c04df09e5b170c11abae62&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:19.168119 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45663fa8-cd11-48bb-bc77-fc974e96056f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:19.168614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45663fa8-cd11-48bb-bc77-fc974e96056f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:19.168614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45663fa8-cd11-48bb-bc77-fc974e96056f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:19.169919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 567/1147] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:19.174929 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44d4e8-0085-46bc-b70d-68b84cd442a5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:19.175293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44d4e8-0085-46bc-b70d-68b84cd442a5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:19.175466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44d4e8-0085-46bc-b70d-68b84cd442a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:19.193522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2f44d4e8-0085-46bc-b70d-68b84cd442a5 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:19.194966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2f44d4e8-0085-46bc-b70d-68b84cd442a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:19.257693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 581/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:19.529882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45663fa8-cd11-48bb-bc77-fc974e96056f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:19.551326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:19.627996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45663fa8-cd11-48bb-bc77-fc974e96056f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:19.659272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a883-0fcb-4bc9-a31d-023194d80b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:19.659876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a883-0fcb-4bc9-a31d-023194d80b43 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:19.660151 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a883-0fcb-4bc9-a31d-023194d80b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:19.660420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a883-0fcb-4bc9-a31d-023194d80b43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:19.670457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98a883-0fcb-4bc9-a31d-023194d80b43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-fa98a883-0fcb-4bc9-a31d-023194d80b43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:26:19.675132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:26:19.676047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:26:19.677043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 568/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:26:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:26:19.682019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44d4e8-0085-46bc-b70d-68b84cd442a5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:19.770524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:19.776161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44d4e8-0085-46bc-b70d-68b84cd442a5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:19.810887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be45509-a0a5-443b-909d-3c6293cd1a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:19.811425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be45509-a0a5-443b-909d-3c6293cd1a06 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:19.811705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be45509-a0a5-443b-909d-3c6293cd1a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:19.811978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be45509-a0a5-443b-909d-3c6293cd1a06 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:19.812636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be45509-a0a5-443b-909d-3c6293cd1a06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:19.812941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be45509-a0a5-443b-909d-3c6293cd1a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:19.814336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be45509-a0a5-443b-909d-3c6293cd1a06 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:19.822554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 582/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:26:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:19.828298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a883-0fcb-4bc9-a31d-023194d80b43 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:19.908831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:19.927195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a883-0fcb-4bc9-a31d-023194d80b43 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:19.959246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc0da3-0ccd-4b27-9ce0-4c577d263f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:19.959862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0da3-0ccd-4b27-9ce0-4c577d263f5a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:19.960140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0da3-0ccd-4b27-9ce0-4c577d263f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:19.960414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0da3-0ccd-4b27-9ce0-4c577d263f5a None admin] PATH_INFO: `/v3/users/af2f5c83eeed4c998cf29f28dec5a1a0` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:19.961514 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc0da3-0ccd-4b27-9ce0-4c577d263f5a None admin] RBAC: Authorizing `identity:get_user(user_id=af2f5c83eeed4c998cf29f28dec5a1a0)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:19.962960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc0da3-0ccd-4b27-9ce0-4c577d263f5a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:19.964511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 569/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:19 2022] GET /identity/v3/users/af2f5c83eeed4c998cf29f28dec5a1a0 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:19.970924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be45509-a0a5-443b-909d-3c6293cd1a06 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:20.068532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be45509-a0a5-443b-909d-3c6293cd1a06 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:20.100811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bbeee-4ec6-47df-81dd-2810160c1163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:20.101432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bbeee-4ec6-47df-81dd-2810160c1163 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:20.101710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bbeee-4ec6-47df-81dd-2810160c1163 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:20.101982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bbeee-4ec6-47df-81dd-2810160c1163 None admin] PATH_INFO: `/v3/projects/6ed26558a9c04df09e5b170c11abae62` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:20.103109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bbeee-4ec6-47df-81dd-2810160c1163 None admin] RBAC: Authorizing `identity:get_project(project_id=6ed26558a9c04df09e5b170c11abae62)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:20.104533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bbeee-4ec6-47df-81dd-2810160c1163 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:20.106146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 583/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:26:19 2022] GET /identity/v3/projects/6ed26558a9c04df09e5b170c11abae62 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:20.112626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc0da3-0ccd-4b27-9ce0-4c577d263f5a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:20.207490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc0da3-0ccd-4b27-9ce0-4c577d263f5a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:20.239897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47046aa3-1bef-45a2-9afd-d7650d5b6461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:20.240530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47046aa3-1bef-45a2-9afd-d7650d5b6461 None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:20.240808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47046aa3-1bef-45a2-9afd-d7650d5b6461 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:20.241079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47046aa3-1bef-45a2-9afd-d7650d5b6461 None admin] PATH_INFO: `/v3/projects/6ed26558a9c04df09e5b170c11abae62/users/af2f5c83eeed4c998cf29f28dec5a1a0/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:20.242973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47046aa3-1bef-45a2-9afd-d7650d5b6461 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ed26558a9c04df09e5b170c11abae62, user_id=af2f5c83eeed4c998cf29f28dec5a1a0, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:20.244368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47046aa3-1bef-45a2-9afd-d7650d5b6461 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:20.340250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 570/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:26:20 2022] PUT /identity/v3/projects/6ed26558a9c04df09e5b170c11abae62/users/af2f5c83eeed4c998cf29f28dec5a1a0/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:26:22.445264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4dc6c-7775-4bcd-8c2d-23a9a26804de None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:22.445814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4dc6c-7775-4bcd-8c2d-23a9a26804de None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:22.445814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4dc6c-7775-4bcd-8c2d-23a9a26804de None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:22.447187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 584/1154] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:22.452297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a9da9-f8b8-4a41-96c5-701fb708a3ed None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:22.452727 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a9da9-f8b8-4a41-96c5-701fb708a3ed None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:22.452877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a9da9-f8b8-4a41-96c5-701fb708a3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:22.470491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-763a9da9-f8b8-4a41-96c5-701fb708a3ed None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:22.471673 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-763a9da9-f8b8-4a41-96c5-701fb708a3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:22.473551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:22.536227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 571/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:22.806293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4dc6c-7775-4bcd-8c2d-23a9a26804de None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:22.834355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:22.906516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4dc6c-7775-4bcd-8c2d-23a9a26804de None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:22.939365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70600eae-d029-4fd0-9cc2-0eaab69b14b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:22.939985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70600eae-d029-4fd0-9cc2-0eaab69b14b5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:22.940259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70600eae-d029-4fd0-9cc2-0eaab69b14b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:22.940527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70600eae-d029-4fd0-9cc2-0eaab69b14b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:22.951181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70600eae-d029-4fd0-9cc2-0eaab69b14b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-70600eae-d029-4fd0-9cc2-0eaab69b14b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:26:22.956066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:26:22.957142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:26:22.957990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 585/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:26:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:26:22.963354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a9da9-f8b8-4a41-96c5-701fb708a3ed None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:23.057369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a9da9-f8b8-4a41-96c5-701fb708a3ed None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:23.090333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd18bd-8600-4470-b216-a34871376d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:23.090960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd18bd-8600-4470-b216-a34871376d6c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:23.091273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd18bd-8600-4470-b216-a34871376d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:23.091537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd18bd-8600-4470-b216-a34871376d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:23.092206 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd18bd-8600-4470-b216-a34871376d6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:23.092502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd18bd-8600-4470-b216-a34871376d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:23.093851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd18bd-8600-4470-b216-a34871376d6c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:23.098854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:23.102110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 572/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:26:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:23.108721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70600eae-d029-4fd0-9cc2-0eaab69b14b5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:23.190711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:23.205185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70600eae-d029-4fd0-9cc2-0eaab69b14b5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:23.237905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c08948-3968-44bb-99f8-3309e37b570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:23.238543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c08948-3968-44bb-99f8-3309e37b570f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:23.238821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c08948-3968-44bb-99f8-3309e37b570f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:23.239092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c08948-3968-44bb-99f8-3309e37b570f None admin] PATH_INFO: `/v3/users/af2f5c83eeed4c998cf29f28dec5a1a0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:23.240193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c08948-3968-44bb-99f8-3309e37b570f None admin] RBAC: Authorizing `identity:get_user(user_id=af2f5c83eeed4c998cf29f28dec5a1a0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:23.241612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c08948-3968-44bb-99f8-3309e37b570f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:23.243177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 586/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:23 2022] GET /identity/v3/users/af2f5c83eeed4c998cf29f28dec5a1a0 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:23.249415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd18bd-8600-4470-b216-a34871376d6c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:23.343735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd18bd-8600-4470-b216-a34871376d6c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:23.375791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488be378-5900-4bdf-b2cf-ab20ad87a7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:23.376375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488be378-5900-4bdf-b2cf-ab20ad87a7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:23.376654 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488be378-5900-4bdf-b2cf-ab20ad87a7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:23.376923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488be378-5900-4bdf-b2cf-ab20ad87a7f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:23.438088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:23.440245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488be378-5900-4bdf-b2cf-ab20ad87a7f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:23.441660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488be378-5900-4bdf-b2cf-ab20ad87a7f8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:23.534541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 573/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:26:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:26:23.540298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c08948-3968-44bb-99f8-3309e37b570f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:23.635069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c08948-3968-44bb-99f8-3309e37b570f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:23.670691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa873f3-ca8d-4508-b21d-73a4d009a4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:23.671308 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa873f3-ca8d-4508-b21d-73a4d009a4dc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:23.671587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa873f3-ca8d-4508-b21d-73a4d009a4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:23.671861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa873f3-ca8d-4508-b21d-73a4d009a4dc None admin] PATH_INFO: `/v3/projects/6ed26558a9c04df09e5b170c11abae62` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:23.672959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa873f3-ca8d-4508-b21d-73a4d009a4dc None admin] RBAC: Authorizing `identity:get_project(project_id=6ed26558a9c04df09e5b170c11abae62)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:23.674403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa873f3-ca8d-4508-b21d-73a4d009a4dc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:23.676002 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 587/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:26:23 2022] GET /identity/v3/projects/6ed26558a9c04df09e5b170c11abae62 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:23.681988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488be378-5900-4bdf-b2cf-ab20ad87a7f8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:23.773286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:23.776062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488be378-5900-4bdf-b2cf-ab20ad87a7f8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:23.808274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b6ca3-094a-40ec-b7bf-6778e84972d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:23.808854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b6ca3-094a-40ec-b7bf-6778e84972d2 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:23.809129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b6ca3-094a-40ec-b7bf-6778e84972d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:23.809424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b6ca3-094a-40ec-b7bf-6778e84972d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:23.810125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b6ca3-094a-40ec-b7bf-6778e84972d2 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=6ed26558a9c04df09e5b170c11abae62, user.id=af2f5c83eeed4c998cf29f28dec5a1a0) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:23.810456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b6ca3-094a-40ec-b7bf-6778e84972d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:23.811924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b6ca3-094a-40ec-b7bf-6778e84972d2 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:23.817565 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 574/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:26:23 2022] GET /identity/v3/role_assignments?user.id=af2f5c83eeed4c998cf29f28dec5a1a0&scope.project.id=6ed26558a9c04df09e5b170c11abae62&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:25.956205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2aae1b-ba80-4437-8f3d-abf802b2054c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:25.956731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2aae1b-ba80-4437-8f3d-abf802b2054c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:25.956731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2aae1b-ba80-4437-8f3d-abf802b2054c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:25.958086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 588/1162] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:25.963158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5812e0-eff0-4ceb-b0af-86fbc911e48c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:25.963592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5812e0-eff0-4ceb-b0af-86fbc911e48c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:25.963714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5812e0-eff0-4ceb-b0af-86fbc911e48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:25.981352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fd5812e0-eff0-4ceb-b0af-86fbc911e48c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:25.982574 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fd5812e0-eff0-4ceb-b0af-86fbc911e48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:26.045930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 575/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:26.321184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2aae1b-ba80-4437-8f3d-abf802b2054c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:26.344595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:26.418160 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2aae1b-ba80-4437-8f3d-abf802b2054c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:26.451158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e03d09-d801-4516-a825-2828552ffa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:26.451754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e03d09-d801-4516-a825-2828552ffa06 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:26.452029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e03d09-d801-4516-a825-2828552ffa06 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:26.452299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e03d09-d801-4516-a825-2828552ffa06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:26.515875 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e03d09-d801-4516-a825-2828552ffa06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:26.517297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e03d09-d801-4516-a825-2828552ffa06 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:26.539722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:26.611497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 589/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:26:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:26:26.617387 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5812e0-eff0-4ceb-b0af-86fbc911e48c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:26.705046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:26.711354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5812e0-eff0-4ceb-b0af-86fbc911e48c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:26.745360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d2ecf0-701d-4b0a-be5b-809ad06ffc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:26.745981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2ecf0-701d-4b0a-be5b-809ad06ffc70 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:26.746291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2ecf0-701d-4b0a-be5b-809ad06ffc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:26.746559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d2ecf0-701d-4b0a-be5b-809ad06ffc70 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:26.752913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2ecf0-701d-4b0a-be5b-809ad06ffc70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:26.754350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d2ecf0-701d-4b0a-be5b-809ad06ffc70 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-e2d2ecf0-701d-4b0a-be5b-809ad06ffc70 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:26:26.765269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:26:26.766510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:26:26.767535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 576/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 05:26:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:26:26.772317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e03d09-d801-4516-a825-2828552ffa06 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:26.867751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e03d09-d801-4516-a825-2828552ffa06 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:26.888952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:26.901111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374ad9d-42c2-4723-be5d-2e8111f941cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:26.901715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374ad9d-42c2-4723-be5d-2e8111f941cc None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:26.901989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374ad9d-42c2-4723-be5d-2e8111f941cc None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:26.902291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374ad9d-42c2-4723-be5d-2e8111f941cc None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:26.903528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374ad9d-42c2-4723-be5d-2e8111f941cc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:26.903837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374ad9d-42c2-4723-be5d-2e8111f941cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:26.905198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374ad9d-42c2-4723-be5d-2e8111f941cc None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:26.912283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 590/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 05:26:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:29.019344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cb401-a9ca-4883-adc0-58603e4f4f73 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:29.019889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cb401-a9ca-4883-adc0-58603e4f4f73 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:29.019889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cb401-a9ca-4883-adc0-58603e4f4f73 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:29.021218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 577/1167] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:29.026611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd521d-5502-4e9e-a508-3a7a549f7d6e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:29.027030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd521d-5502-4e9e-a508-3a7a549f7d6e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:29.027142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd521d-5502-4e9e-a508-3a7a549f7d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:29.044778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8cbd521d-5502-4e9e-a508-3a7a549f7d6e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:29.045953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8cbd521d-5502-4e9e-a508-3a7a549f7d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:29.110531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 591/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:29.378889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cb401-a9ca-4883-adc0-58603e4f4f73 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:29.477866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cb401-a9ca-4883-adc0-58603e4f4f73 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:29.510914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8eb93-5e93-406d-a2c8-4246166a6297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:29.511498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8eb93-5e93-406d-a2c8-4246166a6297 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:29.511774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8eb93-5e93-406d-a2c8-4246166a6297 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:29.512040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8eb93-5e93-406d-a2c8-4246166a6297 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:29.512703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8eb93-5e93-406d-a2c8-4246166a6297 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:29.514038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8eb93-5e93-406d-a2c8-4246166a6297 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:29.573234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 578/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Tue Oct 4 05:26:29 2022] POST /identity/v3/domains => generated 267 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:26:31.704639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6349290-d166-4297-a484-d947d68f8fac None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:31.705206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6349290-d166-4297-a484-d947d68f8fac None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:31.705206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6349290-d166-4297-a484-d947d68f8fac None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:31.706546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 592/1170] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:31.711932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f0435-eeb9-420f-a835-7f674f8bb6f6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:31.712314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f0435-eeb9-420f-a835-7f674f8bb6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:31.712485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f0435-eeb9-420f-a835-7f674f8bb6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:31.730590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-706f0435-eeb9-420f-a835-7f674f8bb6f6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:31.731822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-706f0435-eeb9-420f-a835-7f674f8bb6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:31.740647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:31.795298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 579/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:32.064658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6349290-d166-4297-a484-d947d68f8fac None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:32.161619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6349290-d166-4297-a484-d947d68f8fac None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:32.177672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:32.195032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dca127-6655-4dc6-a1fd-cb5da088d911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:32.195648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dca127-6655-4dc6-a1fd-cb5da088d911 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:32.195924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dca127-6655-4dc6-a1fd-cb5da088d911 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:32.196194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dca127-6655-4dc6-a1fd-cb5da088d911 None admin] PATH_INFO: `/v3/domains/858fe3e203e148a2b73060f4439b6a33` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:32.206080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dca127-6655-4dc6-a1fd-cb5da088d911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=858fe3e203e148a2b73060f4439b6a33)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:32.207675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dca127-6655-4dc6-a1fd-cb5da088d911 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:32.209613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 593/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Tue Oct 4 05:26:32 2022] GET /identity/v3/domains/858fe3e203e148a2b73060f4439b6a33 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:32.215036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f0435-eeb9-420f-a835-7f674f8bb6f6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:32.308073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f0435-eeb9-420f-a835-7f674f8bb6f6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:32.340625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9116219-0c11-45fb-88a0-c4d5b8d59887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:32.341154 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9116219-0c11-45fb-88a0-c4d5b8d59887 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:32.341431 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9116219-0c11-45fb-88a0-c4d5b8d59887 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:32.341700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9116219-0c11-45fb-88a0-c4d5b8d59887 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:32.342372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9116219-0c11-45fb-88a0-c4d5b8d59887 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:32.343776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9116219-0c11-45fb-88a0-c4d5b8d59887 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:32.358562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:32.477933 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 580/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:26:32 2022] POST /identity/v3/projects => generated 361 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:26:34.611169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455735-2b1a-4b7e-8e6f-7277cedbdadc None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:34.611697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455735-2b1a-4b7e-8e6f-7277cedbdadc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:34.611697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20455735-2b1a-4b7e-8e6f-7277cedbdadc None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:34.613507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 594/1174] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:34.618647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74553aa4-8dc4-4ffc-8781-810004f1a173 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:34.619041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74553aa4-8dc4-4ffc-8781-810004f1a173 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:34.619191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74553aa4-8dc4-4ffc-8781-810004f1a173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:34.636661 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-74553aa4-8dc4-4ffc-8781-810004f1a173 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:34.638376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-74553aa4-8dc4-4ffc-8781-810004f1a173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:34.702012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 581/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:34.971123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455735-2b1a-4b7e-8e6f-7277cedbdadc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:35.070296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20455735-2b1a-4b7e-8e6f-7277cedbdadc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:35.102945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a1109-8dbf-4dc9-b2ca-f7d3ed3aae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:35.103565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1109-8dbf-4dc9-b2ca-f7d3ed3aae92 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:35.103842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1109-8dbf-4dc9-b2ca-f7d3ed3aae92 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:35.104111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1109-8dbf-4dc9-b2ca-f7d3ed3aae92 None admin] PATH_INFO: `/v3/domains/858fe3e203e148a2b73060f4439b6a33` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:35.105233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a1109-8dbf-4dc9-b2ca-f7d3ed3aae92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=858fe3e203e148a2b73060f4439b6a33)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:35.107212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a1109-8dbf-4dc9-b2ca-f7d3ed3aae92 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:35.108759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 595/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Tue Oct 4 05:26:34 2022] GET /identity/v3/domains/858fe3e203e148a2b73060f4439b6a33 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:35.114419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74553aa4-8dc4-4ffc-8781-810004f1a173 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:35.209445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74553aa4-8dc4-4ffc-8781-810004f1a173 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:35.233860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:35.242300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0d689-eb32-48f7-b8e3-3d40af310059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:35.242888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d689-eb32-48f7-b8e3-3d40af310059 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:35.243170 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d689-eb32-48f7-b8e3-3d40af310059 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:35.243445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d689-eb32-48f7-b8e3-3d40af310059 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:35.244093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0d689-eb32-48f7-b8e3-3d40af310059 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:35.245511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0d689-eb32-48f7-b8e3-3d40af310059 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:35.339695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 582/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:26:35 2022] POST /identity/v3/users => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:26:37.512505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207548a1-733f-49d0-9f12-aed167de90cf None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:37.513057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207548a1-733f-49d0-9f12-aed167de90cf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:37.513057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207548a1-733f-49d0-9f12-aed167de90cf None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:37.514393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 596/1178] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:37.519544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781d0e7-a861-43b9-9c66-1add4933ee2d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:37.519925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781d0e7-a861-43b9-9c66-1add4933ee2d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:37.520093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781d0e7-a861-43b9-9c66-1add4933ee2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:37.538233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5781d0e7-a861-43b9-9c66-1add4933ee2d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:37.539403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5781d0e7-a861-43b9-9c66-1add4933ee2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:37.602827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 583/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:37.870876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207548a1-733f-49d0-9f12-aed167de90cf None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:37.917745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:38.047309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207548a1-733f-49d0-9f12-aed167de90cf None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:38.080987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9e8b5-ae1d-4a91-92e5-586ce9bdc3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:38.081596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9e8b5-ae1d-4a91-92e5-586ce9bdc3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:38.081869 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9e8b5-ae1d-4a91-92e5-586ce9bdc3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:38.082161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9e8b5-ae1d-4a91-92e5-586ce9bdc3c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:38.091802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9e8b5-ae1d-4a91-92e5-586ce9bdc3c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-dfc9e8b5-ae1d-4a91-92e5-586ce9bdc3c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:26:38.096557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:26:38.097659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:26:38.098946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 597/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:26:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:26:38.103687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781d0e7-a861-43b9-9c66-1add4933ee2d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:38.197400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781d0e7-a861-43b9-9c66-1add4933ee2d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:38.209589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:38.229855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6445ba-130b-4900-8ac4-e6130197294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:38.231100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6445ba-130b-4900-8ac4-e6130197294a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:38.231385 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6445ba-130b-4900-8ac4-e6130197294a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:38.231658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6445ba-130b-4900-8ac4-e6130197294a None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:38.232322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6445ba-130b-4900-8ac4-e6130197294a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:38.232625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6445ba-130b-4900-8ac4-e6130197294a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:38.234071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6445ba-130b-4900-8ac4-e6130197294a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:38.242111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 584/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:26:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:38.248310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9e8b5-ae1d-4a91-92e5-586ce9bdc3c5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:38.345161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9e8b5-ae1d-4a91-92e5-586ce9bdc3c5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:38.347442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:38.378628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b93a2b-cc82-478d-b725-6a568642d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:38.379246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93a2b-cc82-478d-b725-6a568642d165 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:38.379521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93a2b-cc82-478d-b725-6a568642d165 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:38.379791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93a2b-cc82-478d-b725-6a568642d165 None admin] PATH_INFO: `/v3/users/16e3a01483e5478ab2bea5c15f182b82` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:38.392240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b93a2b-cc82-478d-b725-6a568642d165 None admin] RBAC: Authorizing `identity:get_user(user_id=16e3a01483e5478ab2bea5c15f182b82)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:38.393664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b93a2b-cc82-478d-b725-6a568642d165 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:38.395269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 598/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:38 2022] GET /identity/v3/users/16e3a01483e5478ab2bea5c15f182b82 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:38.401067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6445ba-130b-4900-8ac4-e6130197294a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:38.497232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6445ba-130b-4900-8ac4-e6130197294a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:38.529116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d87ed0-b5bd-4fc4-bba7-4ef178e29f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:38.529708 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d87ed0-b5bd-4fc4-bba7-4ef178e29f52 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:38.529986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d87ed0-b5bd-4fc4-bba7-4ef178e29f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:38.530290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d87ed0-b5bd-4fc4-bba7-4ef178e29f52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:38.570352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:38.594539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d87ed0-b5bd-4fc4-bba7-4ef178e29f52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:38.595969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d87ed0-b5bd-4fc4-bba7-4ef178e29f52 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:38.689823 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 585/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:26:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:26:38.695886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b93a2b-cc82-478d-b725-6a568642d165 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:38.790605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b93a2b-cc82-478d-b725-6a568642d165 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:38.823257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ef2eb-84c9-44f2-bd0a-c28ca45081c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:38.823876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ef2eb-84c9-44f2-bd0a-c28ca45081c8 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:38.824749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ef2eb-84c9-44f2-bd0a-c28ca45081c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:38.825024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ef2eb-84c9-44f2-bd0a-c28ca45081c8 None admin] PATH_INFO: `/v3/projects/1193ce44778a4d14850af3609922cf28` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:38.826123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ef2eb-84c9-44f2-bd0a-c28ca45081c8 None admin] RBAC: Authorizing `identity:get_project(project_id=1193ce44778a4d14850af3609922cf28)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:38.827548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ef2eb-84c9-44f2-bd0a-c28ca45081c8 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:38.829454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 599/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:26:38 2022] GET /identity/v3/projects/1193ce44778a4d14850af3609922cf28 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:38.837369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d87ed0-b5bd-4fc4-bba7-4ef178e29f52 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:38.914770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:38.939556 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d87ed0-b5bd-4fc4-bba7-4ef178e29f52 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:38.972168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e90371-572f-48b2-9039-eb47deab97a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:38.972765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e90371-572f-48b2-9039-eb47deab97a0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:38.973052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e90371-572f-48b2-9039-eb47deab97a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:38.973326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e90371-572f-48b2-9039-eb47deab97a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:38.974039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e90371-572f-48b2-9039-eb47deab97a0 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=1193ce44778a4d14850af3609922cf28, user.id=16e3a01483e5478ab2bea5c15f182b82) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:38.974381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e90371-572f-48b2-9039-eb47deab97a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:38.975790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e90371-572f-48b2-9039-eb47deab97a0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:38.981496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 586/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:26:38 2022] GET /identity/v3/role_assignments?user.id=16e3a01483e5478ab2bea5c15f182b82&scope.project.id=1193ce44778a4d14850af3609922cf28&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:41.056812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3055458-539b-4b2d-bb8c-3d8154818cb2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:41.057351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3055458-539b-4b2d-bb8c-3d8154818cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:41.057351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3055458-539b-4b2d-bb8c-3d8154818cb2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:41.058732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 600/1186] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:41.064023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bb122-dff7-4ee7-85f6-f0cb023f6c47 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:41.064437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bb122-dff7-4ee7-85f6-f0cb023f6c47 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:41.064568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bb122-dff7-4ee7-85f6-f0cb023f6c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:41.082672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e64bb122-dff7-4ee7-85f6-f0cb023f6c47 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:41.083839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e64bb122-dff7-4ee7-85f6-f0cb023f6c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:41.147690 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 587/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:41.415563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3055458-539b-4b2d-bb8c-3d8154818cb2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:41.444566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:41.513276 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3055458-539b-4b2d-bb8c-3d8154818cb2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:41.545249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae71826-84f6-46d6-a78f-9272fab4f396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:41.545865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae71826-84f6-46d6-a78f-9272fab4f396 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:41.546162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae71826-84f6-46d6-a78f-9272fab4f396 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:41.546435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae71826-84f6-46d6-a78f-9272fab4f396 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:41.556028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae71826-84f6-46d6-a78f-9272fab4f396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4ae71826-84f6-46d6-a78f-9272fab4f396 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:26:41.561315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:26:41.562422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:26:41.563456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 601/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:26:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:26:41.568466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bb122-dff7-4ee7-85f6-f0cb023f6c47 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:41.632042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:41.663810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bb122-dff7-4ee7-85f6-f0cb023f6c47 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:41.695558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1772c0-f514-43b8-9084-2b99378dff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:41.696088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1772c0-f514-43b8-9084-2b99378dff12 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:41.696363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1772c0-f514-43b8-9084-2b99378dff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:41.696630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1772c0-f514-43b8-9084-2b99378dff12 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:41.697276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1772c0-f514-43b8-9084-2b99378dff12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:41.697573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1772c0-f514-43b8-9084-2b99378dff12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:41.698933 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1772c0-f514-43b8-9084-2b99378dff12 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:41.707406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 588/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:26:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:41.713048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae71826-84f6-46d6-a78f-9272fab4f396 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:41.800095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:41.809842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae71826-84f6-46d6-a78f-9272fab4f396 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:41.843359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f9199-3d91-46dd-b048-129216570d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:41.843974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9199-3d91-46dd-b048-129216570d5f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:41.844253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9199-3d91-46dd-b048-129216570d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:41.844527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9199-3d91-46dd-b048-129216570d5f None admin] PATH_INFO: `/v3/users/16e3a01483e5478ab2bea5c15f182b82` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:41.845633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f9199-3d91-46dd-b048-129216570d5f None admin] RBAC: Authorizing `identity:get_user(user_id=16e3a01483e5478ab2bea5c15f182b82)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:41.847081 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f9199-3d91-46dd-b048-129216570d5f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:41.848623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 602/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:41 2022] GET /identity/v3/users/16e3a01483e5478ab2bea5c15f182b82 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:41.854190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1772c0-f514-43b8-9084-2b99378dff12 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:41.951451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1772c0-f514-43b8-9084-2b99378dff12 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:41.977887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:41.983951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018d6cd-3349-47c9-807a-57b74d9076bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:41.984606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018d6cd-3349-47c9-807a-57b74d9076bc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:41.984886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018d6cd-3349-47c9-807a-57b74d9076bc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:41.985158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018d6cd-3349-47c9-807a-57b74d9076bc None admin] PATH_INFO: `/v3/projects/1193ce44778a4d14850af3609922cf28` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:41.986355 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018d6cd-3349-47c9-807a-57b74d9076bc None admin] RBAC: Authorizing `identity:get_project(project_id=1193ce44778a4d14850af3609922cf28)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:41.987749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018d6cd-3349-47c9-807a-57b74d9076bc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:41.992279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 589/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:26:41 2022] GET /identity/v3/projects/1193ce44778a4d14850af3609922cf28 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:41.995301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f9199-3d91-46dd-b048-129216570d5f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:42.091151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f9199-3d91-46dd-b048-129216570d5f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:42.123974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4028f-9e77-48e5-9d63-882d16c7d9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:42.124617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4028f-9e77-48e5-9d63-882d16c7d9d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:42.124892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4028f-9e77-48e5-9d63-882d16c7d9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:42.125161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4028f-9e77-48e5-9d63-882d16c7d9d7 None admin] PATH_INFO: `/v3/projects/1193ce44778a4d14850af3609922cf28/users/16e3a01483e5478ab2bea5c15f182b82/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:42.127050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4028f-9e77-48e5-9d63-882d16c7d9d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1193ce44778a4d14850af3609922cf28, user_id=16e3a01483e5478ab2bea5c15f182b82, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:42.128470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4028f-9e77-48e5-9d63-882d16c7d9d7 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:42.182727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 603/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:26:41 2022] PUT /identity/v3/projects/1193ce44778a4d14850af3609922cf28/users/16e3a01483e5478ab2bea5c15f182b82/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:26:44.290005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1677f9-63e8-4d45-a191-2f6206aafcc4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:44.290551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1677f9-63e8-4d45-a191-2f6206aafcc4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:44.290551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1677f9-63e8-4d45-a191-2f6206aafcc4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:44.291849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 590/1193] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:26:44.297160 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2196b-7981-4181-8b4f-adc78a4449d9 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:44.297570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2196b-7981-4181-8b4f-adc78a4449d9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:44.297714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2196b-7981-4181-8b4f-adc78a4449d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:44.315353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a7c2196b-7981-4181-8b4f-adc78a4449d9 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:26:44.316537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a7c2196b-7981-4181-8b4f-adc78a4449d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:26:44.329981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:44.381690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 604/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:26:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:26:44.656005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1677f9-63e8-4d45-a191-2f6206aafcc4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:44.752550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1677f9-63e8-4d45-a191-2f6206aafcc4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:44.784403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783ad11-3254-40ab-b487-fa88dce32b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:44.785011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783ad11-3254-40ab-b487-fa88dce32b31 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:44.785290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783ad11-3254-40ab-b487-fa88dce32b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:44.785559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783ad11-3254-40ab-b487-fa88dce32b31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:44.795064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783ad11-3254-40ab-b487-fa88dce32b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-7783ad11-3254-40ab-b487-fa88dce32b31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:26:44.799876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:26:44.800978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:26:44.802223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 591/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:26:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:26:44.806774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2196b-7981-4181-8b4f-adc78a4449d9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:44.904899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2196b-7981-4181-8b4f-adc78a4449d9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:44.937752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519337e-e617-42a4-8fd5-e65638912200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:44.938315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519337e-e617-42a4-8fd5-e65638912200 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:44.938589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519337e-e617-42a4-8fd5-e65638912200 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:44.938856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519337e-e617-42a4-8fd5-e65638912200 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:44.939508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519337e-e617-42a4-8fd5-e65638912200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:44.939804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519337e-e617-42a4-8fd5-e65638912200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:44.941149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519337e-e617-42a4-8fd5-e65638912200 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:44.949101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 605/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:26:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:44.955019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783ad11-3254-40ab-b487-fa88dce32b31 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:45.011297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:45.050434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783ad11-3254-40ab-b487-fa88dce32b31 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:45.082426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240bd56-1568-41d7-b493-c1898ba21801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:45.083030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240bd56-1568-41d7-b493-c1898ba21801 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:45.083304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240bd56-1568-41d7-b493-c1898ba21801 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:45.083572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240bd56-1568-41d7-b493-c1898ba21801 None admin] PATH_INFO: `/v3/users/16e3a01483e5478ab2bea5c15f182b82` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:45.084660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240bd56-1568-41d7-b493-c1898ba21801 None admin] RBAC: Authorizing `identity:get_user(user_id=16e3a01483e5478ab2bea5c15f182b82)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:45.086053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240bd56-1568-41d7-b493-c1898ba21801 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:45.087608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 592/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 05:26:44 2022] GET /identity/v3/users/16e3a01483e5478ab2bea5c15f182b82 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:45.093324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519337e-e617-42a4-8fd5-e65638912200 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:45.106066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:45.189906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519337e-e617-42a4-8fd5-e65638912200 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:45.221670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707b506-e8f8-4bd2-9bc6-e57a67bafb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:45.222297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707b506-e8f8-4bd2-9bc6-e57a67bafb6c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:45.222574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707b506-e8f8-4bd2-9bc6-e57a67bafb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:45.222844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707b506-e8f8-4bd2-9bc6-e57a67bafb6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:45.287109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707b506-e8f8-4bd2-9bc6-e57a67bafb6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:45.288540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707b506-e8f8-4bd2-9bc6-e57a67bafb6c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:45.299642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:45.383617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 606/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:26:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:26:45.389744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240bd56-1568-41d7-b493-c1898ba21801 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:45.487473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240bd56-1568-41d7-b493-c1898ba21801 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:45.511151 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:45.519441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700e42-66e4-425b-ba5f-c93d0c34f5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:45.520064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700e42-66e4-425b-ba5f-c93d0c34f5ae None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:45.520344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700e42-66e4-425b-ba5f-c93d0c34f5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:45.520618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700e42-66e4-425b-ba5f-c93d0c34f5ae None admin] PATH_INFO: `/v3/projects/1193ce44778a4d14850af3609922cf28` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:45.521908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700e42-66e4-425b-ba5f-c93d0c34f5ae None admin] RBAC: Authorizing `identity:get_project(project_id=1193ce44778a4d14850af3609922cf28)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:45.523367 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700e42-66e4-425b-ba5f-c93d0c34f5ae None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:45.524984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 593/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 05:26:45 2022] GET /identity/v3/projects/1193ce44778a4d14850af3609922cf28 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:45.531612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707b506-e8f8-4bd2-9bc6-e57a67bafb6c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:26:45.627289 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707b506-e8f8-4bd2-9bc6-e57a67bafb6c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:26:45.638676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:45.661635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75463c-5ede-4b1a-99d1-f993044f3510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:26:45.662259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75463c-5ede-4b1a-99d1-f993044f3510 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:26:45.662542 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75463c-5ede-4b1a-99d1-f993044f3510 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:26:45.662816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75463c-5ede-4b1a-99d1-f993044f3510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:26:45.663529 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75463c-5ede-4b1a-99d1-f993044f3510 None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=1193ce44778a4d14850af3609922cf28, user.id=16e3a01483e5478ab2bea5c15f182b82) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:26:45.663837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75463c-5ede-4b1a-99d1-f993044f3510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:26:45.665267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75463c-5ede-4b1a-99d1-f993044f3510 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:26:45.670963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 607/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:26:45 2022] GET /identity/v3/role_assignments?user.id=16e3a01483e5478ab2bea5c15f182b82&scope.project.id=1193ce44778a4d14850af3609922cf28&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:26:55.539802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:26:55.674911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=13 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:27:57.863172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b224e7-8ab2-44f5-a179-905b8f32dccd None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:27:57.863757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b224e7-8ab2-44f5-a179-905b8f32dccd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:27:57.863757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b224e7-8ab2-44f5-a179-905b8f32dccd None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:27:57.865041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 594/1201] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:27:57.870247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d81e0-3b22-4b1e-bb34-bdc08813115f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:27:57.870706 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d81e0-3b22-4b1e-bb34-bdc08813115f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:27:57.870806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d81e0-3b22-4b1e-bb34-bdc08813115f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:27:57.890082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-328d81e0-3b22-4b1e-bb34-bdc08813115f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:27:57.890842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-328d81e0-3b22-4b1e-bb34-bdc08813115f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:27:57.956057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 608/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:27:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:27:58.220045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b224e7-8ab2-44f5-a179-905b8f32dccd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:27:58.318324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b224e7-8ab2-44f5-a179-905b8f32dccd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:27:58.351489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3769d-714c-4118-808b-f07bd3f4b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:27:58.352107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3769d-714c-4118-808b-f07bd3f4b09b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:27:58.352380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3769d-714c-4118-808b-f07bd3f4b09b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:27:58.352645 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3769d-714c-4118-808b-f07bd3f4b09b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:27:58.362697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3769d-714c-4118-808b-f07bd3f4b09b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:27:58.364117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3769d-714c-4118-808b-f07bd3f4b09b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-0cc3769d-714c-4118-808b-f07bd3f4b09b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:27:58.378503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:27:58.379744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:27:58.380809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:27:58.380809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:27:58.380809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:27:58.380809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 595/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:27:58.385794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d81e0-3b22-4b1e-bb34-bdc08813115f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:27:58.481934 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d81e0-3b22-4b1e-bb34-bdc08813115f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:27:58.509504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:27:58.515238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e273366-fef3-47ad-807d-810c8b17a8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:27:58.515841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e273366-fef3-47ad-807d-810c8b17a8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:27:58.516116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e273366-fef3-47ad-807d-810c8b17a8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:27:58.516385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e273366-fef3-47ad-807d-810c8b17a8d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:27:58.517046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e273366-fef3-47ad-807d-810c8b17a8d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:27:58.517346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e273366-fef3-47ad-807d-810c8b17a8d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:27:58.518727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e273366-fef3-47ad-807d-810c8b17a8d9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:27:58.530200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 609/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:27:58.535907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3769d-714c-4118-808b-f07bd3f4b09b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:27:58.586832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:27:58.632196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3769d-714c-4118-808b-f07bd3f4b09b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:27:58.668626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab2051-ce0a-4991-9250-78808a8fe75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:27:58.669220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab2051-ce0a-4991-9250-78808a8fe75e None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:27:58.669495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab2051-ce0a-4991-9250-78808a8fe75e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:27:58.669765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab2051-ce0a-4991-9250-78808a8fe75e None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:27:58.670430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab2051-ce0a-4991-9250-78808a8fe75e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:27:58.671890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab2051-ce0a-4991-9250-78808a8fe75e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:27:58.758257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 596/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 05:27:58 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:28:00.939922 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbd520-49d2-49b6-a634-c61f6cbb7ae9 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:00.940424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbd520-49d2-49b6-a634-c61f6cbb7ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:00.940522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbd520-49d2-49b6-a634-c61f6cbb7ae9 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:00.941820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 610/1206] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:28:00.947458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6ff83-f130-46c6-bf1d-9ab085797452 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:00.947873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6ff83-f130-46c6-bf1d-9ab085797452 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:00.948030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6ff83-f130-46c6-bf1d-9ab085797452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:00.966442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-56c6ff83-f130-46c6-bf1d-9ab085797452 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:28:00.967618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-56c6ff83-f130-46c6-bf1d-9ab085797452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:28:01.031638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 597/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:28:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:28:01.300240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbd520-49d2-49b6-a634-c61f6cbb7ae9 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:01.408335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fbd520-49d2-49b6-a634-c61f6cbb7ae9 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:01.441163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccd689-6676-4492-ab75-79a0dd121173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:01.441814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccd689-6676-4492-ab75-79a0dd121173 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:01.442088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccd689-6676-4492-ab75-79a0dd121173 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:01.442380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccd689-6676-4492-ab75-79a0dd121173 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:01.452976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccd689-6676-4492-ab75-79a0dd121173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-beccd689-6676-4492-ab75-79a0dd121173 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:28:01.457821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:01.458873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:01.459732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 611/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:28:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:01.465809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6ff83-f130-46c6-bf1d-9ab085797452 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:01.505019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:01.562392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6ff83-f130-46c6-bf1d-9ab085797452 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:01.596518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68163d9-d26e-4fae-8436-f86ddce104f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:01.597106 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68163d9-d26e-4fae-8436-f86ddce104f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:01.597388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68163d9-d26e-4fae-8436-f86ddce104f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:01.597662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68163d9-d26e-4fae-8436-f86ddce104f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:01.598351 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68163d9-d26e-4fae-8436-f86ddce104f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:01.598658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68163d9-d26e-4fae-8436-f86ddce104f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:01.600006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68163d9-d26e-4fae-8436-f86ddce104f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:01.608501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 598/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:28:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:01.614448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccd689-6676-4492-ab75-79a0dd121173 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:01.664563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:01.713086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccd689-6676-4492-ab75-79a0dd121173 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:01.746526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d683e-ef9c-4ef4-92a2-a990ec0f4c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:01.747149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d683e-ef9c-4ef4-92a2-a990ec0f4c2d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:01.747427 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d683e-ef9c-4ef4-92a2-a990ec0f4c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:01.747698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d683e-ef9c-4ef4-92a2-a990ec0f4c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:01.758010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d683e-ef9c-4ef4-92a2-a990ec0f4c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:01.760052 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d683e-ef9c-4ef4-92a2-a990ec0f4c2d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-8b0d683e-ef9c-4ef4-92a2-a990ec0f4c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:01.773898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:01.775079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:01.776007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:01.776007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:01.776007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:01.776007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 612/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:01.780975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68163d9-d26e-4fae-8436-f86ddce104f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:01.878195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:01.881143 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68163d9-d26e-4fae-8436-f86ddce104f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:01.914961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d98593-fd78-46b4-9f8b-1cfae250c23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:01.915562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d98593-fd78-46b4-9f8b-1cfae250c23f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:01.915841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d98593-fd78-46b4-9f8b-1cfae250c23f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:01.916113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d98593-fd78-46b4-9f8b-1cfae250c23f None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:01.916742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d98593-fd78-46b4-9f8b-1cfae250c23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:01.917041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d98593-fd78-46b4-9f8b-1cfae250c23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:01.918425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d98593-fd78-46b4-9f8b-1cfae250c23f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:01.929077 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 599/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:01.935468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d683e-ef9c-4ef4-92a2-a990ec0f4c2d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:02.033709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d683e-ef9c-4ef4-92a2-a990ec0f4c2d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:02.035241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:02.068005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062aeac-29d8-4a8f-bf3e-b14934cbe04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:02.068876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062aeac-29d8-4a8f-bf3e-b14934cbe04f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:02.069153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062aeac-29d8-4a8f-bf3e-b14934cbe04f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:02.069424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062aeac-29d8-4a8f-bf3e-b14934cbe04f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:02.076741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062aeac-29d8-4a8f-bf3e-b14934cbe04f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:02.078162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062aeac-29d8-4a8f-bf3e-b14934cbe04f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-6062aeac-29d8-4a8f-bf3e-b14934cbe04f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:28:02.089268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:02.090438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:02.091444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 613/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:02.096431 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d98593-fd78-46b4-9f8b-1cfae250c23f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:02.195946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d98593-fd78-46b4-9f8b-1cfae250c23f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:02.228871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654642c-3caa-40ec-98d2-7b5fc4e1a328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:02.229469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654642c-3caa-40ec-98d2-7b5fc4e1a328 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:02.229746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654642c-3caa-40ec-98d2-7b5fc4e1a328 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:02.230057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654642c-3caa-40ec-98d2-7b5fc4e1a328 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:02.230877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654642c-3caa-40ec-98d2-7b5fc4e1a328 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:02.231184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654642c-3caa-40ec-98d2-7b5fc4e1a328 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:02.232586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654642c-3caa-40ec-98d2-7b5fc4e1a328 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:02.250286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 600/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 05:28:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:02.256486 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062aeac-29d8-4a8f-bf3e-b14934cbe04f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:02.356254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062aeac-29d8-4a8f-bf3e-b14934cbe04f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:02.389258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08130f84-71bc-4f67-a559-87c9a60eb6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:02.389844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08130f84-71bc-4f67-a559-87c9a60eb6ee None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:02.390119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08130f84-71bc-4f67-a559-87c9a60eb6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:02.390418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08130f84-71bc-4f67-a559-87c9a60eb6ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:02.419457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:02.458032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08130f84-71bc-4f67-a559-87c9a60eb6ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:02.459445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08130f84-71bc-4f67-a559-87c9a60eb6ee None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:02.554608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 614/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:28:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:28:02.560655 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654642c-3caa-40ec-98d2-7b5fc4e1a328 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:02.579576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:02.657464 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654642c-3caa-40ec-98d2-7b5fc4e1a328 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:02.689893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b4be7-d206-4a06-a697-39e6bd6f4ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:02.690532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b4be7-d206-4a06-a697-39e6bd6f4ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:02.690811 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b4be7-d206-4a06-a697-39e6bd6f4ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:02.691081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b4be7-d206-4a06-a697-39e6bd6f4ae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:02.700897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b4be7-d206-4a06-a697-39e6bd6f4ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:02.707248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b4be7-d206-4a06-a697-39e6bd6f4ae1 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-809b4be7-d206-4a06-a697-39e6bd6f4ae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:02.716801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:02.717716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:02.718912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:02.718912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:02.718912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:02.718912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 601/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:02.723660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08130f84-71bc-4f67-a559-87c9a60eb6ee None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:02.783681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:02.820933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08130f84-71bc-4f67-a559-87c9a60eb6ee None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:02.853903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4ff41-2e60-4176-a870-62b72d1a217c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:02.854627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4ff41-2e60-4176-a870-62b72d1a217c None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:02.854904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4ff41-2e60-4176-a870-62b72d1a217c None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:02.855170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4ff41-2e60-4176-a870-62b72d1a217c None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:02.855790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4ff41-2e60-4176-a870-62b72d1a217c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:02.856088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4ff41-2e60-4176-a870-62b72d1a217c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:02.857471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4ff41-2e60-4176-a870-62b72d1a217c None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:02.869326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 615/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:02.875186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b4be7-d206-4a06-a697-39e6bd6f4ae1 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:02.943208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:02.973835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b4be7-d206-4a06-a697-39e6bd6f4ae1 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:03.005675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f58bd-dabb-46da-932f-e9e72bec6991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:03.006500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f58bd-dabb-46da-932f-e9e72bec6991 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:03.007207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f58bd-dabb-46da-932f-e9e72bec6991 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:03.007480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f58bd-dabb-46da-932f-e9e72bec6991 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:03.014709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f58bd-dabb-46da-932f-e9e72bec6991 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:03.016289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f58bd-dabb-46da-932f-e9e72bec6991 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-370f58bd-dabb-46da-932f-e9e72bec6991 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:28:03.028128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:03.029217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:03.030531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:28:03.030531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:28:03.030531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:28:03.030531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:28:03.030531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:28:03.030531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:28:03.030531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:03.030531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:03.030531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 602/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:28:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:03.035152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4ff41-2e60-4176-a870-62b72d1a217c None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:03.132281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4ff41-2e60-4176-a870-62b72d1a217c None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:03.157409 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:03.168445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71159434-db07-409d-92ee-e6c5cae1d917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:03.169047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71159434-db07-409d-92ee-e6c5cae1d917 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:03.169321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71159434-db07-409d-92ee-e6c5cae1d917 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:03.169591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71159434-db07-409d-92ee-e6c5cae1d917 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:03.170305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71159434-db07-409d-92ee-e6c5cae1d917 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:03.170657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71159434-db07-409d-92ee-e6c5cae1d917 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:03.172167 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71159434-db07-409d-92ee-e6c5cae1d917 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:03.183818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 616/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:28:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:03.189995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f58bd-dabb-46da-932f-e9e72bec6991 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:03.287918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f58bd-dabb-46da-932f-e9e72bec6991 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:03.315347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:03.321249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b80ca-1314-4b39-b174-8f788a845980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:03.322402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b80ca-1314-4b39-b174-8f788a845980 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:03.322687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b80ca-1314-4b39-b174-8f788a845980 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:03.322958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b80ca-1314-4b39-b174-8f788a845980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:03.323674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b80ca-1314-4b39-b174-8f788a845980 None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=e95901cce5d448acb2571976163868a8) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:03.323981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b80ca-1314-4b39-b174-8f788a845980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:03.325399 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b80ca-1314-4b39-b174-8f788a845980 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:03.330707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 603/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:28:03 2022] GET /identity/v3/role_assignments?user.id=e95901cce5d448acb2571976163868a8&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:05.424105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f4f38-4fbe-4945-b21c-754b936d4f79 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:05.424670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f4f38-4fbe-4945-b21c-754b936d4f79 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:05.424670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f4f38-4fbe-4945-b21c-754b936d4f79 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:05.425959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 617/1220] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:28:05.431618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726b1bc-09d9-4bf5-9b1c-d73218e9f0e7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:05.432017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726b1bc-09d9-4bf5-9b1c-d73218e9f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:05.433907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726b1bc-09d9-4bf5-9b1c-d73218e9f0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:05.450990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9726b1bc-09d9-4bf5-9b1c-d73218e9f0e7 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:28:05.452168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9726b1bc-09d9-4bf5-9b1c-d73218e9f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:28:05.515734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 604/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:28:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:28:05.788437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f4f38-4fbe-4945-b21c-754b936d4f79 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:05.890017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f4f38-4fbe-4945-b21c-754b936d4f79 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:05.923686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6c904-35fe-435f-8b15-34e5a62583f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:05.926071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6c904-35fe-435f-8b15-34e5a62583f2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:05.926071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6c904-35fe-435f-8b15-34e5a62583f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:05.926071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6c904-35fe-435f-8b15-34e5a62583f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:05.935402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6c904-35fe-435f-8b15-34e5a62583f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-ead6c904-35fe-435f-8b15-34e5a62583f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:28:05.940248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:05.941242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:05.942202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 618/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:28:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:05.948133 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726b1bc-09d9-4bf5-9b1c-d73218e9f0e7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:06.043853 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726b1bc-09d9-4bf5-9b1c-d73218e9f0e7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:06.066270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:06.077352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b66ee5-27aa-44d1-ab85-0e9e652f5da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:06.077953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66ee5-27aa-44d1-ab85-0e9e652f5da6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:06.078262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66ee5-27aa-44d1-ab85-0e9e652f5da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:06.078537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b66ee5-27aa-44d1-ab85-0e9e652f5da6 None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:06.079260 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b66ee5-27aa-44d1-ab85-0e9e652f5da6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:06.079558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b66ee5-27aa-44d1-ab85-0e9e652f5da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:06.080910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b66ee5-27aa-44d1-ab85-0e9e652f5da6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:06.089328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 605/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:28:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:06.095017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6c904-35fe-435f-8b15-34e5a62583f2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:06.138274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:06.193102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6c904-35fe-435f-8b15-34e5a62583f2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:06.227138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a14548-5374-4452-992b-8e4c981ce079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:06.227786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a14548-5374-4452-992b-8e4c981ce079 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:06.228064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a14548-5374-4452-992b-8e4c981ce079 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:06.228332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a14548-5374-4452-992b-8e4c981ce079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:06.238081 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a14548-5374-4452-992b-8e4c981ce079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:06.239581 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a14548-5374-4452-992b-8e4c981ce079 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-93a14548-5374-4452-992b-8e4c981ce079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:06.254150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:06.260431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:06.261555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:06.261555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:06.261555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:06.261555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 619/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:06.261842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b66ee5-27aa-44d1-ab85-0e9e652f5da6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:06.359790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b66ee5-27aa-44d1-ab85-0e9e652f5da6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:06.392650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653e147-970d-4f4e-a819-a97eb720869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:06.393249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653e147-970d-4f4e-a819-a97eb720869d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:06.393525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653e147-970d-4f4e-a819-a97eb720869d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:06.393795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653e147-970d-4f4e-a819-a97eb720869d None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:06.394460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653e147-970d-4f4e-a819-a97eb720869d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:06.394762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653e147-970d-4f4e-a819-a97eb720869d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:06.396108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653e147-970d-4f4e-a819-a97eb720869d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:06.407317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 606/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:06.413196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a14548-5374-4452-992b-8e4c981ce079 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:06.504907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:06.512210 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a14548-5374-4452-992b-8e4c981ce079 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:06.544868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243315e8-8a5a-43a3-bd85-62ef164c71ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:06.545488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243315e8-8a5a-43a3-bd85-62ef164c71ae None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:06.545766 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243315e8-8a5a-43a3-bd85-62ef164c71ae None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:06.546041 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243315e8-8a5a-43a3-bd85-62ef164c71ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:06.552848 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243315e8-8a5a-43a3-bd85-62ef164c71ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:06.554330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243315e8-8a5a-43a3-bd85-62ef164c71ae None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-243315e8-8a5a-43a3-bd85-62ef164c71ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:28:06.566987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:06.568127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:06.569392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 620/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:06.575041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653e147-970d-4f4e-a819-a97eb720869d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:06.627011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:06.677418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653e147-970d-4f4e-a819-a97eb720869d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:06.710082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdde3c-847c-4709-b6e7-9de3b24caf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:06.710715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdde3c-847c-4709-b6e7-9de3b24caf6a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:06.710995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdde3c-847c-4709-b6e7-9de3b24caf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:06.711269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdde3c-847c-4709-b6e7-9de3b24caf6a None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:06.712016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdde3c-847c-4709-b6e7-9de3b24caf6a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:06.712321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdde3c-847c-4709-b6e7-9de3b24caf6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:06.713788 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdde3c-847c-4709-b6e7-9de3b24caf6a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:06.730755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 607/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 05:28:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:06.736625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243315e8-8a5a-43a3-bd85-62ef164c71ae None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:06.833567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243315e8-8a5a-43a3-bd85-62ef164c71ae None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:06.866207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e279d-b7e9-4b91-a212-995a2f0d78ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:06.866821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e279d-b7e9-4b91-a212-995a2f0d78ee None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:06.867095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e279d-b7e9-4b91-a212-995a2f0d78ee None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:06.867360 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e279d-b7e9-4b91-a212-995a2f0d78ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:06.878253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e279d-b7e9-4b91-a212-995a2f0d78ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:06.879667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e279d-b7e9-4b91-a212-995a2f0d78ee None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-b25e279d-b7e9-4b91-a212-995a2f0d78ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:06.894047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:06.900623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:06.901743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:06.901743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:06.901743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:06.901743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 621/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:06.902052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdde3c-847c-4709-b6e7-9de3b24caf6a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:06.999164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdde3c-847c-4709-b6e7-9de3b24caf6a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:07.001393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:07.031693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b7a21-a20d-4cc1-9726-b4d68e73c4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:07.032280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b7a21-a20d-4cc1-9726-b4d68e73c4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:07.032554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b7a21-a20d-4cc1-9726-b4d68e73c4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:07.032922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b7a21-a20d-4cc1-9726-b4d68e73c4d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:07.033516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b7a21-a20d-4cc1-9726-b4d68e73c4d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:07.033812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b7a21-a20d-4cc1-9726-b4d68e73c4d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:07.035195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b7a21-a20d-4cc1-9726-b4d68e73c4d8 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:07.046912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 608/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:07.052647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e279d-b7e9-4b91-a212-995a2f0d78ee None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:07.062335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:07.148996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e279d-b7e9-4b91-a212-995a2f0d78ee None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:07.182494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28819f-162f-447e-996a-77867c0d43fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:07.183113 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28819f-162f-447e-996a-77867c0d43fe None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:07.183386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28819f-162f-447e-996a-77867c0d43fe None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:07.183652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28819f-162f-447e-996a-77867c0d43fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:07.190168 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28819f-162f-447e-996a-77867c0d43fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:07.191608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28819f-162f-447e-996a-77867c0d43fe None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-7f28819f-162f-447e-996a-77867c0d43fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:28:07.202826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:07.203904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:07.205134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:28:07.205134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:28:07.205134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:28:07.205134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:28:07.205134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:28:07.205134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:28:07.205134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:07.205134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:07.205134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 622/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:28:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:07.209716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b7a21-a20d-4cc1-9726-b4d68e73c4d8 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:07.308066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b7a21-a20d-4cc1-9726-b4d68e73c4d8 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:07.340677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6943d8-0972-410c-b987-6056644acc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:07.341268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6943d8-0972-410c-b987-6056644acc6b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:07.341544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6943d8-0972-410c-b987-6056644acc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:07.341903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6943d8-0972-410c-b987-6056644acc6b None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:07.342613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6943d8-0972-410c-b987-6056644acc6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:07.342915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6943d8-0972-410c-b987-6056644acc6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:07.349184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6943d8-0972-410c-b987-6056644acc6b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:07.355575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 609/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:28:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:07.361679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28819f-162f-447e-996a-77867c0d43fe None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:07.427015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:07.462705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28819f-162f-447e-996a-77867c0d43fe None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:07.494503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1ca8c-b92d-4bd8-b802-e15b01c9b0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:07.495211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1ca8c-b92d-4bd8-b802-e15b01c9b0d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:07.495494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1ca8c-b92d-4bd8-b802-e15b01c9b0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:07.495768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1ca8c-b92d-4bd8-b802-e15b01c9b0d3 None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/e95901cce5d448acb2571976163868a8/roles/bc27c203c0234a6194c03f93ee3c50a9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:07.509984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1ca8c-b92d-4bd8-b802-e15b01c9b0d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=e95901cce5d448acb2571976163868a8, role_id=bc27c203c0234a6194c03f93ee3c50a9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:07.511459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1ca8c-b92d-4bd8-b802-e15b01c9b0d3 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:07.615359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 623/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:28:07 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/e95901cce5d448acb2571976163868a8/roles/bc27c203c0234a6194c03f93ee3c50a9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:28:09.734505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd5700-a304-4f76-a4c2-1129f6b7b14b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:09.735070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd5700-a304-4f76-a4c2-1129f6b7b14b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:09.735070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd5700-a304-4f76-a4c2-1129f6b7b14b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:09.736417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 610/1233] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:28:09.741547 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be4620-4956-416c-a523-65d058d8841b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:09.741966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be4620-4956-416c-a523-65d058d8841b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:09.742099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be4620-4956-416c-a523-65d058d8841b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:09.760603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-41be4620-4956-416c-a523-65d058d8841b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:28:09.761909 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-41be4620-4956-416c-a523-65d058d8841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:28:09.827055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 624/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:28:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:28:10.096000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd5700-a304-4f76-a4c2-1129f6b7b14b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:10.121030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:10.193987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd5700-a304-4f76-a4c2-1129f6b7b14b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:10.227852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc9b26-01b9-4cf0-9fd0-7a24f419fb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:10.228477 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc9b26-01b9-4cf0-9fd0-7a24f419fb09 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:10.228759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc9b26-01b9-4cf0-9fd0-7a24f419fb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:10.229031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc9b26-01b9-4cf0-9fd0-7a24f419fb09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:10.238896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc9b26-01b9-4cf0-9fd0-7a24f419fb09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-1ecc9b26-01b9-4cf0-9fd0-7a24f419fb09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:28:10.243737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:10.244909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:10.246090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 611/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 05:28:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:10.250926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be4620-4956-416c-a523-65d058d8841b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:10.297902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:10.350259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be4620-4956-416c-a523-65d058d8841b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:10.382835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d95662-2f8e-4c78-9b0e-dfd8bc17cf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:10.383905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d95662-2f8e-4c78-9b0e-dfd8bc17cf78 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:10.384190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d95662-2f8e-4c78-9b0e-dfd8bc17cf78 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:10.384461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d95662-2f8e-4c78-9b0e-dfd8bc17cf78 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:10.385116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d95662-2f8e-4c78-9b0e-dfd8bc17cf78 None admin] RBAC: Adding query filter params (name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:10.385416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d95662-2f8e-4c78-9b0e-dfd8bc17cf78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:10.386813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d95662-2f8e-4c78-9b0e-dfd8bc17cf78 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:10.395185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 625/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 05:28:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:10.401828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc9b26-01b9-4cf0-9fd0-7a24f419fb09 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:10.485680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:10.501269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc9b26-01b9-4cf0-9fd0-7a24f419fb09 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:10.534906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ee4c6-0431-4f25-974f-45d81fc3055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:10.535547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ee4c6-0431-4f25-974f-45d81fc3055f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:10.535809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ee4c6-0431-4f25-974f-45d81fc3055f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:10.536083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ee4c6-0431-4f25-974f-45d81fc3055f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:10.545750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ee4c6-0431-4f25-974f-45d81fc3055f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:10.547269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ee4c6-0431-4f25-974f-45d81fc3055f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-655ee4c6-0431-4f25-974f-45d81fc3055f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:10.561989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:10.563158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:10.564270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:10.564270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:10.564270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:10.564270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 612/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:10.569616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d95662-2f8e-4c78-9b0e-dfd8bc17cf78 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:10.668258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d95662-2f8e-4c78-9b0e-dfd8bc17cf78 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:10.676641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:10.701516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d99c9-f4b3-4342-bb5c-acac42336dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:10.702109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d99c9-f4b3-4342-bb5c-acac42336dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:10.702422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d99c9-f4b3-4342-bb5c-acac42336dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:10.702698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d99c9-f4b3-4342-bb5c-acac42336dd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:10.703335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d99c9-f4b3-4342-bb5c-acac42336dd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:10.703640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d99c9-f4b3-4342-bb5c-acac42336dd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:10.704994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d99c9-f4b3-4342-bb5c-acac42336dd6 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:10.716089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 626/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:10.721985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ee4c6-0431-4f25-974f-45d81fc3055f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:10.825424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ee4c6-0431-4f25-974f-45d81fc3055f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:10.857802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e521e-92ce-450a-aafa-81720f856bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:10.858455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e521e-92ce-450a-aafa-81720f856bda None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:10.858731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e521e-92ce-450a-aafa-81720f856bda None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:10.858999 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e521e-92ce-450a-aafa-81720f856bda None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:10.861555 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:10.866352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e521e-92ce-450a-aafa-81720f856bda None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:10.878436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e521e-92ce-450a-aafa-81720f856bda None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-c29e521e-92ce-450a-aafa-81720f856bda None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:28:10.879084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:10.880190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:10.881455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 613/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:10.886752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d99c9-f4b3-4342-bb5c-acac42336dd6 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:10.983213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d99c9-f4b3-4342-bb5c-acac42336dd6 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:11.016761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2bc6d-9312-43b7-af50-ddae995b94ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:11.017561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2bc6d-9312-43b7-af50-ddae995b94ff None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:11.017841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2bc6d-9312-43b7-af50-ddae995b94ff None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:11.018117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2bc6d-9312-43b7-af50-ddae995b94ff None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:11.018878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2bc6d-9312-43b7-af50-ddae995b94ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:11.019188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2bc6d-9312-43b7-af50-ddae995b94ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:11.020611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2bc6d-9312-43b7-af50-ddae995b94ff None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:11.037507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 627/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 05:28:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:11.043466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e521e-92ce-450a-aafa-81720f856bda None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:11.141070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e521e-92ce-450a-aafa-81720f856bda None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:11.174016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b9cc0-eea5-482e-a6cc-bdf5d03b8ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:11.174644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b9cc0-eea5-482e-a6cc-bdf5d03b8ddc None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:11.174922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b9cc0-eea5-482e-a6cc-bdf5d03b8ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:11.175234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b9cc0-eea5-482e-a6cc-bdf5d03b8ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:11.233488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:11.239774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b9cc0-eea5-482e-a6cc-bdf5d03b8ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:11.241203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b9cc0-eea5-482e-a6cc-bdf5d03b8ddc None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:11.334789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 614/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:28:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:28:11.340659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2bc6d-9312-43b7-af50-ddae995b94ff None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:11.367304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:11.438215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2bc6d-9312-43b7-af50-ddae995b94ff None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:11.471970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d462318-dd6b-4f5f-ab37-49be81373e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:11.472591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d462318-dd6b-4f5f-ab37-49be81373e7d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:11.472863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d462318-dd6b-4f5f-ab37-49be81373e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:11.473131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d462318-dd6b-4f5f-ab37-49be81373e7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:11.483669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d462318-dd6b-4f5f-ab37-49be81373e7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:11.485121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d462318-dd6b-4f5f-ab37-49be81373e7d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-0d462318-dd6b-4f5f-ab37-49be81373e7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:11.500694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:11.505302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:11.506455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:11.506455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:11.506455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:11.506455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 628/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:11.508047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b9cc0-eea5-482e-a6cc-bdf5d03b8ddc None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:11.601221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:11.607856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b9cc0-eea5-482e-a6cc-bdf5d03b8ddc None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:11.640682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27db68-1ccb-4d38-b6d7-445b3d73ac7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:11.641288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27db68-1ccb-4d38-b6d7-445b3d73ac7d None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:11.641571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27db68-1ccb-4d38-b6d7-445b3d73ac7d None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:11.641847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27db68-1ccb-4d38-b6d7-445b3d73ac7d None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:11.642526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27db68-1ccb-4d38-b6d7-445b3d73ac7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:11.642837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27db68-1ccb-4d38-b6d7-445b3d73ac7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:11.644196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27db68-1ccb-4d38-b6d7-445b3d73ac7d None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:11.655955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 615/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:11.661626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d462318-dd6b-4f5f-ab37-49be81373e7d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:11.740569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:11.760919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d462318-dd6b-4f5f-ab37-49be81373e7d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:11.797666 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddb62a-6542-4130-a4eb-139f9c736f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:11.798321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddb62a-6542-4130-a4eb-139f9c736f88 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:11.798829 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddb62a-6542-4130-a4eb-139f9c736f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:11.798923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddb62a-6542-4130-a4eb-139f9c736f88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:11.805398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddb62a-6542-4130-a4eb-139f9c736f88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:11.806874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddb62a-6542-4130-a4eb-139f9c736f88 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-53ddb62a-6542-4130-a4eb-139f9c736f88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:28:11.817806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:11.819151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:11.820265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:28:11.820265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:28:11.820265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:28:11.820265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:28:11.820265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:28:11.820265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:28:11.820265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:11.820265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:11.820265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 629/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:28:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:11.825465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27db68-1ccb-4d38-b6d7-445b3d73ac7d None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:11.923317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27db68-1ccb-4d38-b6d7-445b3d73ac7d None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:11.956480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db38ab-f2a8-40cf-82f1-5d929dfadf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:11.957102 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db38ab-f2a8-40cf-82f1-5d929dfadf11 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:11.957381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db38ab-f2a8-40cf-82f1-5d929dfadf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:11.957651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db38ab-f2a8-40cf-82f1-5d929dfadf11 None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:11.958386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db38ab-f2a8-40cf-82f1-5d929dfadf11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:11.958691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db38ab-f2a8-40cf-82f1-5d929dfadf11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:11.960098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db38ab-f2a8-40cf-82f1-5d929dfadf11 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:11.972058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 616/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:28:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:11.985853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddb62a-6542-4130-a4eb-139f9c736f88 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:12.075228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddb62a-6542-4130-a4eb-139f9c736f88 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:12.107639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598fc80-03a0-4135-ab1c-e49b3c421c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:12.108243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598fc80-03a0-4135-ab1c-e49b3c421c31 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:12.108509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598fc80-03a0-4135-ab1c-e49b3c421c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:12.108892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598fc80-03a0-4135-ab1c-e49b3c421c31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:12.109606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598fc80-03a0-4135-ab1c-e49b3c421c31 None admin] RBAC: Adding query filter params (role.id=bc27c203c0234a6194c03f93ee3c50a9, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=e95901cce5d448acb2571976163868a8) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:12.109911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598fc80-03a0-4135-ab1c-e49b3c421c31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:12.111356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598fc80-03a0-4135-ab1c-e49b3c421c31 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:12.117023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 630/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:28:11 2022] GET /identity/v3/role_assignments?user.id=e95901cce5d448acb2571976163868a8&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=bc27c203c0234a6194c03f93ee3c50a9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:14.317618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c708a-2c0d-48dd-b91a-5c39a8f14f30 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:14.318114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c708a-2c0d-48dd-b91a-5c39a8f14f30 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:14.318114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c708a-2c0d-48dd-b91a-5c39a8f14f30 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:14.319707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 617/1247] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:28:14.325830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070227a-4662-43c3-86c4-ce512a3c81ae None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:14.326241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070227a-4662-43c3-86c4-ce512a3c81ae None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:14.328803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070227a-4662-43c3-86c4-ce512a3c81ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:14.332573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:14.344242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1070227a-4662-43c3-86c4-ce512a3c81ae None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:28:14.345409 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1070227a-4662-43c3-86c4-ce512a3c81ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:28:14.411011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 631/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:28:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:28:14.681328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c708a-2c0d-48dd-b91a-5c39a8f14f30 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:14.699328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:14.781512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c708a-2c0d-48dd-b91a-5c39a8f14f30 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:14.814380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa7011-dad7-45b0-9eee-55d42aaff362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:14.814997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa7011-dad7-45b0-9eee-55d42aaff362 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:14.815271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa7011-dad7-45b0-9eee-55d42aaff362 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:14.815537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa7011-dad7-45b0-9eee-55d42aaff362 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:14.825943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa7011-dad7-45b0-9eee-55d42aaff362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-22fa7011-dad7-45b0-9eee-55d42aaff362 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:28:14.830676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:14.831905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:14.833007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 618/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:28:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:14.838416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070227a-4662-43c3-86c4-ce512a3c81ae None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:14.935670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070227a-4662-43c3-86c4-ce512a3c81ae None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:14.963480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:14.968259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347a149-95fe-4698-9c78-1f4c1ecf86df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:14.968862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347a149-95fe-4698-9c78-1f4c1ecf86df None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:14.969141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347a149-95fe-4698-9c78-1f4c1ecf86df None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:14.969408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347a149-95fe-4698-9c78-1f4c1ecf86df None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:14.970059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347a149-95fe-4698-9c78-1f4c1ecf86df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:14.970390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347a149-95fe-4698-9c78-1f4c1ecf86df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:14.971740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347a149-95fe-4698-9c78-1f4c1ecf86df None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:14.979853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 632/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:28:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:14.985693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa7011-dad7-45b0-9eee-55d42aaff362 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:15.062759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:15.085266 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa7011-dad7-45b0-9eee-55d42aaff362 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:15.118723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d22b3-05ca-41b1-9e11-1537a2d65970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:15.119339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d22b3-05ca-41b1-9e11-1537a2d65970 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:15.119664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d22b3-05ca-41b1-9e11-1537a2d65970 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:15.119944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d22b3-05ca-41b1-9e11-1537a2d65970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:15.130122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d22b3-05ca-41b1-9e11-1537a2d65970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:15.135910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d22b3-05ca-41b1-9e11-1537a2d65970 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-836d22b3-05ca-41b1-9e11-1537a2d65970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:15.145635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:15.150468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:15.151593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:15.151593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:15.151593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:15.151593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 619/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:15.152693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347a149-95fe-4698-9c78-1f4c1ecf86df None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:15.249579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347a149-95fe-4698-9c78-1f4c1ecf86df None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:15.281601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606d16-772a-4ea7-8ce7-4e3eb6dc16a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:15.282220 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606d16-772a-4ea7-8ce7-4e3eb6dc16a5 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:15.282501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606d16-772a-4ea7-8ce7-4e3eb6dc16a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:15.282775 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606d16-772a-4ea7-8ce7-4e3eb6dc16a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:15.283407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4606d16-772a-4ea7-8ce7-4e3eb6dc16a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:15.283711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4606d16-772a-4ea7-8ce7-4e3eb6dc16a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:15.285097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4606d16-772a-4ea7-8ce7-4e3eb6dc16a5 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:15.296950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 633/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:28:15.302700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d22b3-05ca-41b1-9e11-1537a2d65970 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:15.400380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d22b3-05ca-41b1-9e11-1537a2d65970 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:15.425926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:15.433389 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab15c33-ba26-4761-ba43-e499d3b0d7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:15.434025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab15c33-ba26-4761-ba43-e499d3b0d7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:15.434320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab15c33-ba26-4761-ba43-e499d3b0d7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:15.434594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab15c33-ba26-4761-ba43-e499d3b0d7f6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:15.441749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab15c33-ba26-4761-ba43-e499d3b0d7f6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:15.443205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab15c33-ba26-4761-ba43-e499d3b0d7f6 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-dab15c33-ba26-4761-ba43-e499d3b0d7f6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:28:15.454707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:15.455855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:15.457187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 620/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:15.462054 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606d16-772a-4ea7-8ce7-4e3eb6dc16a5 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:15.516608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:15.560155 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606d16-772a-4ea7-8ce7-4e3eb6dc16a5 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:15.594298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f3bbc-9834-4217-b748-5de712348dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:15.594890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f3bbc-9834-4217-b748-5de712348dbe None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:15.595172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f3bbc-9834-4217-b748-5de712348dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:15.595443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f3bbc-9834-4217-b748-5de712348dbe None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:15.596163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f3bbc-9834-4217-b748-5de712348dbe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:15.596468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f3bbc-9834-4217-b748-5de712348dbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:15.597900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f3bbc-9834-4217-b748-5de712348dbe None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:15.616055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 634/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 05:28:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:15.621963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab15c33-ba26-4761-ba43-e499d3b0d7f6 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:15.719366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab15c33-ba26-4761-ba43-e499d3b0d7f6 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:15.751117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2748a-5ce4-4046-8843-ca40062e5e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:15.751713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2748a-5ce4-4046-8843-ca40062e5e2b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:15.751993 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2748a-5ce4-4046-8843-ca40062e5e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:15.752269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2748a-5ce4-4046-8843-ca40062e5e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:15.807231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:15.819091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2748a-5ce4-4046-8843-ca40062e5e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:15.820526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2748a-5ce4-4046-8843-ca40062e5e2b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:15.917392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 621/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:28:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:28:15.923176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f3bbc-9834-4217-b748-5de712348dbe None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:16.026374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f3bbc-9834-4217-b748-5de712348dbe None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:16.039168 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:16.060064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c85b42-9f80-42b8-a517-0f17d211c250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:16.060642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c85b42-9f80-42b8-a517-0f17d211c250 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:16.060920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c85b42-9f80-42b8-a517-0f17d211c250 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:16.061371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c85b42-9f80-42b8-a517-0f17d211c250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:16.072053 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c85b42-9f80-42b8-a517-0f17d211c250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:16.073613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c85b42-9f80-42b8-a517-0f17d211c250 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c2c85b42-9f80-42b8-a517-0f17d211c250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:16.087820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:16.088758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:16.089734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:16.089734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:16.089734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:16.089734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 635/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:16.094738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2748a-5ce4-4046-8843-ca40062e5e2b None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:16.181388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:16.192520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2748a-5ce4-4046-8843-ca40062e5e2b None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:16.225321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cc3ce-7479-4ba9-a5c3-f455512461de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:16.225915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cc3ce-7479-4ba9-a5c3-f455512461de None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:16.226226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cc3ce-7479-4ba9-a5c3-f455512461de None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:16.226496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cc3ce-7479-4ba9-a5c3-f455512461de None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:16.227152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cc3ce-7479-4ba9-a5c3-f455512461de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:16.227448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cc3ce-7479-4ba9-a5c3-f455512461de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:16.228785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cc3ce-7479-4ba9-a5c3-f455512461de None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:16.239500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 622/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:16.245749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c85b42-9f80-42b8-a517-0f17d211c250 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:16.341999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c85b42-9f80-42b8-a517-0f17d211c250 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:16.374391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba30764-ba05-4d19-8fd0-027856ad66dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:16.374995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba30764-ba05-4d19-8fd0-027856ad66dd None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:16.375270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba30764-ba05-4d19-8fd0-027856ad66dd None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:16.378778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba30764-ba05-4d19-8fd0-027856ad66dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:16.381857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba30764-ba05-4d19-8fd0-027856ad66dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:16.383352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba30764-ba05-4d19-8fd0-027856ad66dd None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-4ba30764-ba05-4d19-8fd0-027856ad66dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:28:16.394406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:16.395439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:16.396621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:28:16.396621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:28:16.396621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:28:16.396621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:28:16.396621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:28:16.396621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:28:16.396621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:16.396621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:16.396621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 636/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:28:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:16.401291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cc3ce-7479-4ba9-a5c3-f455512461de None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:16.499659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cc3ce-7479-4ba9-a5c3-f455512461de None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:16.531816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d2ffa-6173-4a3f-bba6-dd25509f0cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:16.532421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d2ffa-6173-4a3f-bba6-dd25509f0cab None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:16.532696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d2ffa-6173-4a3f-bba6-dd25509f0cab None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:16.532966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d2ffa-6173-4a3f-bba6-dd25509f0cab None admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:16.533688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d2ffa-6173-4a3f-bba6-dd25509f0cab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:16.533993 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d2ffa-6173-4a3f-bba6-dd25509f0cab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:16.535529 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d2ffa-6173-4a3f-bba6-dd25509f0cab None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:16.548178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 623/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:28:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:16.554305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba30764-ba05-4d19-8fd0-027856ad66dd None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:16.580346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:16.653521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba30764-ba05-4d19-8fd0-027856ad66dd None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:16.688030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3f849-abb2-41ab-a2cf-516c8ad8215d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:16.688630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3f849-abb2-41ab-a2cf-516c8ad8215d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:16.688906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3f849-abb2-41ab-a2cf-516c8ad8215d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:16.689176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3f849-abb2-41ab-a2cf-516c8ad8215d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:16.689896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3f849-abb2-41ab-a2cf-516c8ad8215d None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=e95901cce5d448acb2571976163868a8) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:16.690234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3f849-abb2-41ab-a2cf-516c8ad8215d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:16.691669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3f849-abb2-41ab-a2cf-516c8ad8215d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:16.697183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 637/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:28:16 2022] GET /identity/v3/role_assignments?user.id=e95901cce5d448acb2571976163868a8&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:18.816225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd7b4e-b204-4749-9c55-2cc09569cd70 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:18.816782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd7b4e-b204-4749-9c55-2cc09569cd70 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:18.816782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd7b4e-b204-4749-9c55-2cc09569cd70 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:18.818156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 624/1261] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:28:18.823981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da51ad-6c00-4ef7-af2d-09817db81ed1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:18.824415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da51ad-6c00-4ef7-af2d-09817db81ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:18.824505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da51ad-6c00-4ef7-af2d-09817db81ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:18.846864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-69da51ad-6c00-4ef7-af2d-09817db81ed1 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:28:18.848055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-69da51ad-6c00-4ef7-af2d-09817db81ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:28:18.914915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 638/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:28:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:28:19.178628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd7b4e-b204-4749-9c55-2cc09569cd70 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:19.189813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:19.279892 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd7b4e-b204-4749-9c55-2cc09569cd70 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:19.312550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f77bc-c6fb-4f51-9c4e-f480b4782eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:19.313169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f77bc-c6fb-4f51-9c4e-f480b4782eac None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:19.313447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f77bc-c6fb-4f51-9c4e-f480b4782eac None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:19.313715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f77bc-c6fb-4f51-9c4e-f480b4782eac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:19.324304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f77bc-c6fb-4f51-9c4e-f480b4782eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-ec9f77bc-c6fb-4f51-9c4e-f480b4782eac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:28:19.329073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:19.330225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:19.331510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 625/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:28:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:19.336278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da51ad-6c00-4ef7-af2d-09817db81ed1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:19.354304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:19.430923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da51ad-6c00-4ef7-af2d-09817db81ed1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:19.462450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308953fe-a7d3-489c-8adc-b1ab8780a2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:19.463051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308953fe-a7d3-489c-8adc-b1ab8780a2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:19.467246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308953fe-a7d3-489c-8adc-b1ab8780a2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:19.467246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308953fe-a7d3-489c-8adc-b1ab8780a2d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:19.467246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308953fe-a7d3-489c-8adc-b1ab8780a2d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:19.467246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308953fe-a7d3-489c-8adc-b1ab8780a2d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:19.467246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308953fe-a7d3-489c-8adc-b1ab8780a2d2 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:19.473520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 639/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:28:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:19.479883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f77bc-c6fb-4f51-9c4e-f480b4782eac None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:19.545031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:19.578588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f77bc-c6fb-4f51-9c4e-f480b4782eac None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:19.611369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8b48f-d18c-4d0c-8572-78f57f32c45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:19.612358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8b48f-d18c-4d0c-8572-78f57f32c45e None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:19.612641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8b48f-d18c-4d0c-8572-78f57f32c45e None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:19.612910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8b48f-d18c-4d0c-8572-78f57f32c45e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:19.623889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8b48f-d18c-4d0c-8572-78f57f32c45e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:19.625327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8b48f-d18c-4d0c-8572-78f57f32c45e None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-40c8b48f-d18c-4d0c-8572-78f57f32c45e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:19.640943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:19.643950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:19.645079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:19.645079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:19.645079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:19.645079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 626/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:19.647824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308953fe-a7d3-489c-8adc-b1ab8780a2d2 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:19.727302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:19.745869 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308953fe-a7d3-489c-8adc-b1ab8780a2d2 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:19.779924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94e4c2-ddf5-4b0a-8644-a1fd593ac879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:19.780544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94e4c2-ddf5-4b0a-8644-a1fd593ac879 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:19.780805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94e4c2-ddf5-4b0a-8644-a1fd593ac879 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:19.781082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94e4c2-ddf5-4b0a-8644-a1fd593ac879 None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:19.781715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94e4c2-ddf5-4b0a-8644-a1fd593ac879 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:19.782017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94e4c2-ddf5-4b0a-8644-a1fd593ac879 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:19.783406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94e4c2-ddf5-4b0a-8644-a1fd593ac879 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:19.794881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 640/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:19.800781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8b48f-d18c-4d0c-8572-78f57f32c45e None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:19.900727 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8b48f-d18c-4d0c-8572-78f57f32c45e None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:19.920725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:19.934003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3712be-a091-4e47-a7df-1a18bb48d510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:19.934659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3712be-a091-4e47-a7df-1a18bb48d510 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:19.934940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3712be-a091-4e47-a7df-1a18bb48d510 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:19.935211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3712be-a091-4e47-a7df-1a18bb48d510 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:19.944012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3712be-a091-4e47-a7df-1a18bb48d510 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:19.945469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3712be-a091-4e47-a7df-1a18bb48d510 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-bb3712be-a091-4e47-a7df-1a18bb48d510 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:28:19.956371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:19.957500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:19.958783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 627/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:28:19.966253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94e4c2-ddf5-4b0a-8644-a1fd593ac879 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:20.064736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94e4c2-ddf5-4b0a-8644-a1fd593ac879 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:20.097917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd6e82-1ea4-40cd-a9fa-98c5cde7acb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:20.098543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd6e82-1ea4-40cd-a9fa-98c5cde7acb9 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:20.098816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd6e82-1ea4-40cd-a9fa-98c5cde7acb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:20.099079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd6e82-1ea4-40cd-a9fa-98c5cde7acb9 None admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:20.099790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd6e82-1ea4-40cd-a9fa-98c5cde7acb9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:20.100107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd6e82-1ea4-40cd-a9fa-98c5cde7acb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:20.101588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd6e82-1ea4-40cd-a9fa-98c5cde7acb9 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:20.110227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:20.117906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 641/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 05:28:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:28:20.123872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3712be-a091-4e47-a7df-1a18bb48d510 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:20.223310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3712be-a091-4e47-a7df-1a18bb48d510 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:20.256313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec962cf-c777-4154-80ae-c2484c46ae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:20.256939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec962cf-c777-4154-80ae-c2484c46ae50 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:20.257219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec962cf-c777-4154-80ae-c2484c46ae50 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:20.257492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec962cf-c777-4154-80ae-c2484c46ae50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:20.268207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec962cf-c777-4154-80ae-c2484c46ae50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:20.269636 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec962cf-c777-4154-80ae-c2484c46ae50 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-4ec962cf-c777-4154-80ae-c2484c46ae50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:20.283921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:20.285775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:20.286737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:20.286737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:20.286737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:20.286737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 628/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:20.291068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd6e82-1ea4-40cd-a9fa-98c5cde7acb9 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:20.387959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd6e82-1ea4-40cd-a9fa-98c5cde7acb9 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:20.421647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d86c2-e438-438f-b68a-eb8730f0443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:20.422270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d86c2-e438-438f-b68a-eb8730f0443d None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:20.422546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d86c2-e438-438f-b68a-eb8730f0443d None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:20.422816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d86c2-e438-438f-b68a-eb8730f0443d None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:20.423442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d86c2-e438-438f-b68a-eb8730f0443d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:20.423738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d86c2-e438-438f-b68a-eb8730f0443d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:20.425092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d86c2-e438-438f-b68a-eb8730f0443d None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:20.436828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 642/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:20.442787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec962cf-c777-4154-80ae-c2484c46ae50 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:20.474063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:20.539358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec962cf-c777-4154-80ae-c2484c46ae50 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:20.572127 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b053664-1869-43bf-8b0d-a133e8e4e3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:20.572743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b053664-1869-43bf-8b0d-a133e8e4e3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:20.573024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b053664-1869-43bf-8b0d-a133e8e4e3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:20.573297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b053664-1869-43bf-8b0d-a133e8e4e3c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:20.579622 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b053664-1869-43bf-8b0d-a133e8e4e3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:20.581030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b053664-1869-43bf-8b0d-a133e8e4e3c0 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-2b053664-1869-43bf-8b0d-a133e8e4e3c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:28:20.591876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:20.594029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:20.595032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:28:20.595032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:28:20.595032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:28:20.595032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:28:20.595032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:28:20.595032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:28:20.595032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:20.595032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:20.595032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 629/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:28:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 05:28:20.599018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d86c2-e438-438f-b68a-eb8730f0443d None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:20.656647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:20.697402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d86c2-e438-438f-b68a-eb8730f0443d None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:20.729397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b664e-d343-41b7-943e-3989e28a8078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:20.730303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b664e-d343-41b7-943e-3989e28a8078 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:20.730591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b664e-d343-41b7-943e-3989e28a8078 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:20.730858 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b664e-d343-41b7-943e-3989e28a8078 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:20.731550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b664e-d343-41b7-943e-3989e28a8078 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:20.731855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b664e-d343-41b7-943e-3989e28a8078 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:20.733295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b664e-d343-41b7-943e-3989e28a8078 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:20.745657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 643/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:28:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:20.751741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b053664-1869-43bf-8b0d-a133e8e4e3c0 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:20.840299 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:20.848681 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b053664-1869-43bf-8b0d-a133e8e4e3c0 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:20.882326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031efbe-1097-40e2-bbd3-71fd175d181a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:20.882989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031efbe-1097-40e2-bbd3-71fd175d181a None admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:20.883267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031efbe-1097-40e2-bbd3-71fd175d181a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:20.883553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031efbe-1097-40e2-bbd3-71fd175d181a None admin] PATH_INFO: `/v3/projects/e74d59a086274094806c1c6cf2964451/users/e95901cce5d448acb2571976163868a8/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:20.885413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031efbe-1097-40e2-bbd3-71fd175d181a None admin] RBAC: Authorizing `identity:create_grant(project_id=e74d59a086274094806c1c6cf2964451, user_id=e95901cce5d448acb2571976163868a8, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:20.886876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031efbe-1097-40e2-bbd3-71fd175d181a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:21.005147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 630/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 05:28:20 2022] PUT /identity/v3/projects/e74d59a086274094806c1c6cf2964451/users/e95901cce5d448acb2571976163868a8/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:28:23.125342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3237fb-d141-4e59-a2c4-d6eb17e505e4 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:23.125890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3237fb-d141-4e59-a2c4-d6eb17e505e4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:23.125890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3237fb-d141-4e59-a2c4-d6eb17e505e4 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:23.127253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 644/1274] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:28:23.132465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34a0d0-f74d-4cef-9249-376e95d2f185 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:23.132932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34a0d0-f74d-4cef-9249-376e95d2f185 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:23.134803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34a0d0-f74d-4cef-9249-376e95d2f185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:23.152303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bd34a0d0-f74d-4cef-9249-376e95d2f185 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:28:23.153404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bd34a0d0-f74d-4cef-9249-376e95d2f185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:28:23.218056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 631/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:28:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:28:23.488710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3237fb-d141-4e59-a2c4-d6eb17e505e4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:23.589928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3237fb-d141-4e59-a2c4-d6eb17e505e4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:23.606758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:23.624048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbffa4-18bf-47dd-a8b9-46beb6ad1005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:23.624676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbffa4-18bf-47dd-a8b9-46beb6ad1005 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:23.624955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbffa4-18bf-47dd-a8b9-46beb6ad1005 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:23.625223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbffa4-18bf-47dd-a8b9-46beb6ad1005 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:23.636383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbffa4-18bf-47dd-a8b9-46beb6ad1005 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-c1fbffa4-18bf-47dd-a8b9-46beb6ad1005 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 05:28:23.641330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:23.642467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:23.643469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 645/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 05:28:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:23.648506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34a0d0-f74d-4cef-9249-376e95d2f185 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:23.728557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:23.746254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34a0d0-f74d-4cef-9249-376e95d2f185 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:23.778550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3b4a6-1432-4051-9f02-36f181eefb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:23.779144 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3b4a6-1432-4051-9f02-36f181eefb9f None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:23.779423 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3b4a6-1432-4051-9f02-36f181eefb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:23.779693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3b4a6-1432-4051-9f02-36f181eefb9f None admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:23.780344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3b4a6-1432-4051-9f02-36f181eefb9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:23.780642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3b4a6-1432-4051-9f02-36f181eefb9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:23.782075 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3b4a6-1432-4051-9f02-36f181eefb9f None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:23.790551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 632/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 05:28:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:23.796464 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbffa4-18bf-47dd-a8b9-46beb6ad1005 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:23.896849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbffa4-18bf-47dd-a8b9-46beb6ad1005 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:23.931033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e029c-8b80-48d4-9a51-949f4e029c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:23.931681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e029c-8b80-48d4-9a51-949f4e029c15 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:23.931954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e029c-8b80-48d4-9a51-949f4e029c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:23.932221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e029c-8b80-48d4-9a51-949f4e029c15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:23.942076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e029c-8b80-48d4-9a51-949f4e029c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:23.943799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e029c-8b80-48d4-9a51-949f4e029c15 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-720e029c-8b80-48d4-9a51-949f4e029c15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:23.957965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:23.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:23.960455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:23.960455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:23.960455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:23.960455 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 646/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:23.965162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3b4a6-1432-4051-9f02-36f181eefb9f None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:24.063686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3b4a6-1432-4051-9f02-36f181eefb9f None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:24.095822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cfccc-7627-43f4-9934-977ee8224db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:24.096421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cfccc-7627-43f4-9934-977ee8224db5 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:24.096696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cfccc-7627-43f4-9934-977ee8224db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:24.096972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cfccc-7627-43f4-9934-977ee8224db5 None admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:24.097596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cfccc-7627-43f4-9934-977ee8224db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:24.097897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cfccc-7627-43f4-9934-977ee8224db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:24.099320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cfccc-7627-43f4-9934-977ee8224db5 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:24.111161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 633/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:24.116978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e029c-8b80-48d4-9a51-949f4e029c15 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:24.147545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:24.218572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e029c-8b80-48d4-9a51-949f4e029c15 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:24.247954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c264fa-86a2-422d-9abf-2559dc6df353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:24.248568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c264fa-86a2-422d-9abf-2559dc6df353 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:24.248843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c264fa-86a2-422d-9abf-2559dc6df353 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:24.249110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c264fa-86a2-422d-9abf-2559dc6df353 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:24.255732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c264fa-86a2-422d-9abf-2559dc6df353 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:24.257156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c264fa-86a2-422d-9abf-2559dc6df353 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-10c264fa-86a2-422d-9abf-2559dc6df353 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 05:28:24.269192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:24.274468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:24.275755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 647/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:24.276830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cfccc-7627-43f4-9934-977ee8224db5 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:24.280782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:24.374510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cfccc-7627-43f4-9934-977ee8224db5 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:24.408825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf993ddd-a25e-4cdf-80a5-f41084ae5311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:24.409417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993ddd-a25e-4cdf-80a5-f41084ae5311 None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:24.409693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993ddd-a25e-4cdf-80a5-f41084ae5311 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:24.409963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993ddd-a25e-4cdf-80a5-f41084ae5311 None admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:24.410708 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf993ddd-a25e-4cdf-80a5-f41084ae5311 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:24.411015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf993ddd-a25e-4cdf-80a5-f41084ae5311 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:24.412417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf993ddd-a25e-4cdf-80a5-f41084ae5311 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:24.429315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 634/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 05:28:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:28:24.435258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c264fa-86a2-422d-9abf-2559dc6df353 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:24.521426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:24.531424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c264fa-86a2-422d-9abf-2559dc6df353 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:24.564123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f620902-c688-4698-ab95-40982554c19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:24.564802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f620902-c688-4698-ab95-40982554c19f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:24.565077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f620902-c688-4698-ab95-40982554c19f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:24.565347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f620902-c688-4698-ab95-40982554c19f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:24.632728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f620902-c688-4698-ab95-40982554c19f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:24.634650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f620902-c688-4698-ab95-40982554c19f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:24.731867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:24.742030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 648/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 05:28:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 05:28:24.748162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf993ddd-a25e-4cdf-80a5-f41084ae5311 None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:24.815750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:24.858091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf993ddd-a25e-4cdf-80a5-f41084ae5311 None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:24.899941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c46667-4339-430f-a037-eafb1250cc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:24.900597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c46667-4339-430f-a037-eafb1250cc0a None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:24.900875 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c46667-4339-430f-a037-eafb1250cc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:24.901199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c46667-4339-430f-a037-eafb1250cc0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:24.913970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c46667-4339-430f-a037-eafb1250cc0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:24.915551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c46667-4339-430f-a037-eafb1250cc0a None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-f3c46667-4339-430f-a037-eafb1250cc0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 05:28:24.931752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:24.933002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 05:28:24.934189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 05:28:24.934189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 05:28:24.934189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:24.937895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 635/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:28:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:24.939086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f620902-c688-4698-ab95-40982554c19f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:25.043044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f620902-c688-4698-ab95-40982554c19f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:25.079459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998fdf4-5ed2-44d8-a49d-4f62c873074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:25.080152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998fdf4-5ed2-44d8-a49d-4f62c873074f None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:25.080445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998fdf4-5ed2-44d8-a49d-4f62c873074f None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:25.080714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998fdf4-5ed2-44d8-a49d-4f62c873074f None admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:25.081392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998fdf4-5ed2-44d8-a49d-4f62c873074f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:25.081695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998fdf4-5ed2-44d8-a49d-4f62c873074f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:25.083094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998fdf4-5ed2-44d8-a49d-4f62c873074f None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:25.094498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 649/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 05:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:25.100263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c46667-4339-430f-a037-eafb1250cc0a None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:25.203046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c46667-4339-430f-a037-eafb1250cc0a None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:25.216071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:25.235027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ddf5d-df7c-4edf-b52e-26f5f439a97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:25.235733 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ddf5d-df7c-4edf-b52e-26f5f439a97c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:25.236013 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ddf5d-df7c-4edf-b52e-26f5f439a97c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:25.236284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ddf5d-df7c-4edf-b52e-26f5f439a97c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:25.243745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ddf5d-df7c-4edf-b52e-26f5f439a97c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:25.245253 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ddf5d-df7c-4edf-b52e-26f5f439a97c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application [None req-5b9ddf5d-df7c-4edf-b52e-26f5f439a97c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 05:28:25.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:25.258117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:25.259891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 05:28:25.259891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 05:28:25.259891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 05:28:25.259891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 05:28:25.259891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 05:28:25.259891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 05:28:25.259891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 05:28:25.259891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: ERROR keystone.server.flask.application Oct 04 05:28:25.259891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 636/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 05:28:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:25.264603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998fdf4-5ed2-44d8-a49d-4f62c873074f None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:25.312301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:25.359990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998fdf4-5ed2-44d8-a49d-4f62c873074f None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:25.392514 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db001d-f1ac-44da-a755-a5c7df390bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:25.393111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db001d-f1ac-44da-a755-a5c7df390bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:25.393382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db001d-f1ac-44da-a755-a5c7df390bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:25.393648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db001d-f1ac-44da-a755-a5c7df390bc1 None admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:25.394350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db001d-f1ac-44da-a755-a5c7df390bc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:25.394649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db001d-f1ac-44da-a755-a5c7df390bc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:25.396062 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db001d-f1ac-44da-a755-a5c7df390bc1 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:25.407558 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 650/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 05:28:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:25.413714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ddf5d-df7c-4edf-b52e-26f5f439a97c None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:25.510719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ddf5d-df7c-4edf-b52e-26f5f439a97c None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:25.543498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76542a11-3e62-4b12-bf0f-47fbc3d7430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:25.544093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76542a11-3e62-4b12-bf0f-47fbc3d7430c None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:25.544371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76542a11-3e62-4b12-bf0f-47fbc3d7430c None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:25.544645 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76542a11-3e62-4b12-bf0f-47fbc3d7430c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:25.545354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76542a11-3e62-4b12-bf0f-47fbc3d7430c None admin] RBAC: Adding query filter params (role.id=d0a94c98a6c84c18a651e6b8ad809d04, scope.project.id=e74d59a086274094806c1c6cf2964451, user.id=e95901cce5d448acb2571976163868a8) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:25.545662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76542a11-3e62-4b12-bf0f-47fbc3d7430c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:25.547095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76542a11-3e62-4b12-bf0f-47fbc3d7430c None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:25.552676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 637/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 05:28:25 2022] GET /identity/v3/role_assignments?user.id=e95901cce5d448acb2571976163868a8&scope.project.id=e74d59a086274094806c1c6cf2964451&role.id=d0a94c98a6c84c18a651e6b8ad809d04 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:27.725035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146f291-019e-4d7d-a589-f66bbd4c5a07 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:27.725642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146f291-019e-4d7d-a589-f66bbd4c5a07 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:27.725642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146f291-019e-4d7d-a589-f66bbd4c5a07 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:27.726962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 651/1288] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:28:27.732320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f492735-30b3-4295-b696-3f0c1109870b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:27.732775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f492735-30b3-4295-b696-3f0c1109870b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:27.732874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f492735-30b3-4295-b696-3f0c1109870b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:27.750543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6f492735-30b3-4295-b696-3f0c1109870b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:28:27.751725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6f492735-30b3-4295-b696-3f0c1109870b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:28:27.781496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:27.816740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 638/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:28:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:28:28.086590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146f291-019e-4d7d-a589-f66bbd4c5a07 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:28.186060 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146f291-019e-4d7d-a589-f66bbd4c5a07 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:28.197296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:28.219430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43692312-19a3-45a6-a063-6b58979b71d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:28.220056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43692312-19a3-45a6-a063-6b58979b71d0 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:28.220335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43692312-19a3-45a6-a063-6b58979b71d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:28.220609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43692312-19a3-45a6-a063-6b58979b71d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:28.221184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43692312-19a3-45a6-a063-6b58979b71d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:28.222581 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43692312-19a3-45a6-a063-6b58979b71d0 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-43692312-19a3-45a6-a063-6b58979b71d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:28.231032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:28.232347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:28.233347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 652/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 05:28:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:28.238370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f492735-30b3-4295-b696-3f0c1109870b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:28.335625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f492735-30b3-4295-b696-3f0c1109870b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:28.368816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120b685-dab6-4486-b716-da4c144234ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:28.376067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120b685-dab6-4486-b716-da4c144234ff None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:28.376067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120b685-dab6-4486-b716-da4c144234ff None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:28.376067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120b685-dab6-4486-b716-da4c144234ff None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:28.376067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120b685-dab6-4486-b716-da4c144234ff None admin] RBAC: Adding query filter params (name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:28.376067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120b685-dab6-4486-b716-da4c144234ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:28.376067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120b685-dab6-4486-b716-da4c144234ff None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:28.376949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 639/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Oct 4 05:28:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:28.382803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43692312-19a3-45a6-a063-6b58979b71d0 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:28.481350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43692312-19a3-45a6-a063-6b58979b71d0 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:28.514205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b26c4-78ad-4714-a47d-b96f27d6a593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:28.514804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b26c4-78ad-4714-a47d-b96f27d6a593 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:28.515096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b26c4-78ad-4714-a47d-b96f27d6a593 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:28.515357 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b26c4-78ad-4714-a47d-b96f27d6a593 None admin] PATH_INFO: `/v3/services` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:28.515979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b26c4-78ad-4714-a47d-b96f27d6a593 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:28.516282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b26c4-78ad-4714-a47d-b96f27d6a593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:28.517633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b26c4-78ad-4714-a47d-b96f27d6a593 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:28.522069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 653/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Oct 4 05:28:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:30.626132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a5528-4336-45c8-b4a5-53c2562c50b8 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:30.626722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a5528-4336-45c8-b4a5-53c2562c50b8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:30.626722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a5528-4336-45c8-b4a5-53c2562c50b8 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:30.628069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 640/1293] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:28:30.633198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b390d-3158-4deb-aa21-9d5af3c5f6c2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:30.633617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b390d-3158-4deb-aa21-9d5af3c5f6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:30.633720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b390d-3158-4deb-aa21-9d5af3c5f6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:30.651465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-001b390d-3158-4deb-aa21-9d5af3c5f6c2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:28:30.652639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-001b390d-3158-4deb-aa21-9d5af3c5f6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:28:30.681686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:30.716745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 654/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:28:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 05:28:30.982408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a5528-4336-45c8-b4a5-53c2562c50b8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:31.012409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:31.079263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a5528-4336-45c8-b4a5-53c2562c50b8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:31.113637 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9b01-fdff-4e1c-a2ff-d1b7065c0c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:31.114279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9b01-fdff-4e1c-a2ff-d1b7065c0c16 None admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:31.114559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9b01-fdff-4e1c-a2ff-d1b7065c0c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:31.114829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9b01-fdff-4e1c-a2ff-d1b7065c0c16 None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:31.115469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9b01-fdff-4e1c-a2ff-d1b7065c0c16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:31.116878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9b01-fdff-4e1c-a2ff-d1b7065c0c16 None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:31.255023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 641/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 05:28:30 2022] POST /identity/v3/services => generated 246 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:28:33.393436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decada8-c126-4269-985a-a92d5361c723 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:33.393970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decada8-c126-4269-985a-a92d5361c723 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:33.393970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decada8-c126-4269-985a-a92d5361c723 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:33.395342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 655/1296] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:28:33.401095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cb533-f4a3-4c08-b163-cdcaece7992c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:33.405369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cb533-f4a3-4c08-b163-cdcaece7992c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:33.405369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cb533-f4a3-4c08-b163-cdcaece7992c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:33.420617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-315cb533-f4a3-4c08-b163-cdcaece7992c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:28:33.421825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-315cb533-f4a3-4c08-b163-cdcaece7992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:28:33.493152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 642/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:28:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:28:33.756995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decada8-c126-4269-985a-a92d5361c723 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:33.856134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decada8-c126-4269-985a-a92d5361c723 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:33.890991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e210bc4-31ec-478b-b5ee-e47622ed87ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:33.891621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e210bc4-31ec-478b-b5ee-e47622ed87ae None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:33.891896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e210bc4-31ec-478b-b5ee-e47622ed87ae None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:33.892161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e210bc4-31ec-478b-b5ee-e47622ed87ae None admin] PATH_INFO: `/v3/services/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:33.892738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e210bc4-31ec-478b-b5ee-e47622ed87ae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:33.894089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e210bc4-31ec-478b-b5ee-e47622ed87ae None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-9e210bc4-31ec-478b-b5ee-e47622ed87ae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:33.903453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:33.904622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:33.905496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 656/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 05:28:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:33.910496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cb533-f4a3-4c08-b163-cdcaece7992c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:33.931330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:34.008775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cb533-f4a3-4c08-b163-cdcaece7992c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:34.041243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9552045-6132-4be6-b187-6feb95cc26cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:34.041840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9552045-6132-4be6-b187-6feb95cc26cb None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:34.042117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9552045-6132-4be6-b187-6feb95cc26cb None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:34.042422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9552045-6132-4be6-b187-6feb95cc26cb None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:34.043040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9552045-6132-4be6-b187-6feb95cc26cb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:34.043342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9552045-6132-4be6-b187-6feb95cc26cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:34.044691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9552045-6132-4be6-b187-6feb95cc26cb None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:34.049722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 643/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Oct 4 05:28:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:34.056213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e210bc4-31ec-478b-b5ee-e47622ed87ae None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:34.088351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:34.155355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e210bc4-31ec-478b-b5ee-e47622ed87ae None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:34.188710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037859b-1b51-4aca-b0d5-af0745319909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:34.189313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037859b-1b51-4aca-b0d5-af0745319909 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:34.189590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037859b-1b51-4aca-b0d5-af0745319909 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:34.189859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037859b-1b51-4aca-b0d5-af0745319909 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:34.190575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037859b-1b51-4aca-b0d5-af0745319909 None admin] RBAC: Adding query filter params (interface=public, service_id=c512dc0b773749dbaf896a180a6b3834, region_id=RegionOne) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:34.190880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037859b-1b51-4aca-b0d5-af0745319909 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:34.192237 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037859b-1b51-4aca-b0d5-af0745319909 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:34.197935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 657/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 05:28:34 2022] GET /identity/v3/endpoints?service_id=c512dc0b773749dbaf896a180a6b3834&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:34.203795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9552045-6132-4be6-b187-6feb95cc26cb None admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:34.282297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:34.301072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9552045-6132-4be6-b187-6feb95cc26cb None admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:34.334959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31483112-4e06-4968-8c33-3f81b14fba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:34.335645 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31483112-4e06-4968-8c33-3f81b14fba9b None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:34.335923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31483112-4e06-4968-8c33-3f81b14fba9b None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:34.336190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31483112-4e06-4968-8c33-3f81b14fba9b None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:34.336783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31483112-4e06-4968-8c33-3f81b14fba9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:34.338113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31483112-4e06-4968-8c33-3f81b14fba9b None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:34.345339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 644/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 05:28:34 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:28:36.449514 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ba630-12ac-4699-ad7d-6609b2b10105 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:36.450088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ba630-12ac-4699-ad7d-6609b2b10105 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:36.450088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ba630-12ac-4699-ad7d-6609b2b10105 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:36.451419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 658/1302] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:28:36.456560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8603f6-0804-4a53-98a3-6548bea7f6d9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:36.456947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8603f6-0804-4a53-98a3-6548bea7f6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:36.457118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8603f6-0804-4a53-98a3-6548bea7f6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:36.475319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1b8603f6-0804-4a53-98a3-6548bea7f6d9 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:28:36.477130 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1b8603f6-0804-4a53-98a3-6548bea7f6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:28:36.540438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 645/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:28:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 05:28:36.807097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ba630-12ac-4699-ad7d-6609b2b10105 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:36.898130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:36.904646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ba630-12ac-4699-ad7d-6609b2b10105 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:36.937707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3425b8-7a93-4217-9045-139bec0ec174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:36.938351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3425b8-7a93-4217-9045-139bec0ec174 None admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:36.938625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3425b8-7a93-4217-9045-139bec0ec174 None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:36.938892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3425b8-7a93-4217-9045-139bec0ec174 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:36.939472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3425b8-7a93-4217-9045-139bec0ec174 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:36.941249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3425b8-7a93-4217-9045-139bec0ec174 None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application [None req-db3425b8-7a93-4217-9045-139bec0ec174 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 05:28:36.950710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 05:28:36.953178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application with Lock( Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self._enter() Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application return self.creator() Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application created_value = creator( Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: ERROR keystone.server.flask.application Oct 04 05:28:36.954259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 659/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 05:28:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 05:28:36.957948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8603f6-0804-4a53-98a3-6548bea7f6d9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:37.032423 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:37.056661 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8603f6-0804-4a53-98a3-6548bea7f6d9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:37.092746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf985c7-fc7f-4337-840f-b333db5832ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:37.092746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf985c7-fc7f-4337-840f-b333db5832ba None admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:37.092746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf985c7-fc7f-4337-840f-b333db5832ba None admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:37.092746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf985c7-fc7f-4337-840f-b333db5832ba None admin] PATH_INFO: `/v3/services` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:37.093155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf985c7-fc7f-4337-840f-b333db5832ba None admin] RBAC: Adding query filter params (name=placement) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:28:37.093155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf985c7-fc7f-4337-840f-b333db5832ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:37.094095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf985c7-fc7f-4337-840f-b333db5832ba None admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:37.099080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 646/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Oct 4 05:28:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:28:37.105418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3425b8-7a93-4217-9045-139bec0ec174 None admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:28:37.202000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3425b8-7a93-4217-9045-139bec0ec174 None admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:28:37.235109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c02edf-7881-4373-bbd7-f0b4cd036ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:28:37.235720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c02edf-7881-4373-bbd7-f0b4cd036ede None admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:28:37.236001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c02edf-7881-4373-bbd7-f0b4cd036ede None admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:28:37.236281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c02edf-7881-4373-bbd7-f0b4cd036ede None admin] PATH_INFO: `/v3/endpoints` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:28:37.236952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c02edf-7881-4373-bbd7-f0b4cd036ede None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:28:37.238342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c02edf-7881-4373-bbd7-f0b4cd036ede None admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:28:37.359247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 660/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:28:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:28:47.096114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=30 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:28:47.354850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:31.001445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00e51-885a-4214-b26a-8b16866457e2 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:31.002370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00e51-885a-4214-b26a-8b16866457e2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:31.002862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00e51-885a-4214-b26a-8b16866457e2 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:31.004971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 647/1307] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:34:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:31.010705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6f686-9bf0-43bc-a160-b837987a450f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:31.011152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6f686-9bf0-43bc-a160-b837987a450f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:31.011243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6f686-9bf0-43bc-a160-b837987a450f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:31.091505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e3c6f686-9bf0-43bc-a160-b837987a450f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:31.092748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e3c6f686-9bf0-43bc-a160-b837987a450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:31.207284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 661/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:31.501136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48285b-b4b3-4d5c-b491-032259637e62 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:31.501506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48285b-b4b3-4d5c-b491-032259637e62 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:31.501868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48285b-b4b3-4d5c-b491-032259637e62 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:31.503675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 648/1309] 10.222.0.26 () {54 vars in 964 bytes} [Tue Oct 4 05:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:31.509553 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de772925-f864-48b2-8345-132c37910757 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:31.509871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de772925-f864-48b2-8345-132c37910757 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:31.510122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de772925-f864-48b2-8345-132c37910757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:31.595523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-de772925-f864-48b2-8345-132c37910757 None None] MFA Rules not processed for user `3a0feaf83ee14071806f36d01c36b635`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:31.596908 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-de772925-f864-48b2-8345-132c37910757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:31.621710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:31.683592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 662/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Tue Oct 4 05:34:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:31.693240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48285b-b4b3-4d5c-b491-032259637e62 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:34:31.695053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5a48285b-b4b3-4d5c-b491-032259637e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:31.820948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48285b-b4b3-4d5c-b491-032259637e62 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:34:31.909109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497ffb7-9ffc-4c29-bf0b-232805d22b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:34:31.909845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497ffb7-9ffc-4c29-bf0b-232805d22b44 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:31.910221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497ffb7-9ffc-4c29-bf0b-232805d22b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:31.911098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497ffb7-9ffc-4c29-bf0b-232805d22b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:31.912804 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2497ffb7-9ffc-4c29-bf0b-232805d22b44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:31.976555 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:32.035458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497ffb7-9ffc-4c29-bf0b-232805d22b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:34:32.037936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497ffb7-9ffc-4c29-bf0b-232805d22b44 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:34:32.059625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 649/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:34:34.562041 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dd631-ad4f-47a3-95a0-3c4760712b6c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:34.562604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dd631-ad4f-47a3-95a0-3c4760712b6c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:34.562604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dd631-ad4f-47a3-95a0-3c4760712b6c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:34.563965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 663/1312] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:34.569539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bd2d0-9ec3-421e-92bb-1f10966cacd5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:34.569929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bd2d0-9ec3-421e-92bb-1f10966cacd5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:34.570097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bd2d0-9ec3-421e-92bb-1f10966cacd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:34.654292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6b0bd2d0-9ec3-421e-92bb-1f10966cacd5 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:34.655530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6b0bd2d0-9ec3-421e-92bb-1f10966cacd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:34.751983 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 650/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:35.039250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dd631-ad4f-47a3-95a0-3c4760712b6c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:34:35.040813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-629dd631-ad4f-47a3-95a0-3c4760712b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:35.142071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dd631-ad4f-47a3-95a0-3c4760712b6c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:34:35.212127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:35.214590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d42b9-fe7c-41ff-80c9-57f461536c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:34:35.215195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d42b9-fe7c-41ff-80c9-57f461536c4c service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:35.215472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d42b9-fe7c-41ff-80c9-57f461536c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:35.215742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d42b9-fe7c-41ff-80c9-57f461536c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:35.217342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-028d42b9-fe7c-41ff-80c9-57f461536c4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:35.323695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d42b9-fe7c-41ff-80c9-57f461536c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:34:35.325061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d42b9-fe7c-41ff-80c9-57f461536c4c service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:34:35.343105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 664/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:34:38.089702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ea865-a759-4f81-9538-dba868de4d37 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:38.090305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ea865-a759-4f81-9538-dba868de4d37 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:38.090305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ea865-a759-4f81-9538-dba868de4d37 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:38.091659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 651/1315] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:38.096935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecbda8-7645-4f7c-8d21-3d51f54b3707 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:38.097371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecbda8-7645-4f7c-8d21-3d51f54b3707 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:38.097473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecbda8-7645-4f7c-8d21-3d51f54b3707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:38.161524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c2ecbda8-7645-4f7c-8d21-3d51f54b3707 None None] MFA Rules not processed for user `f0f33c461e67433593c4ca55f81f4e7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:38.162672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c2ecbda8-7645-4f7c-8d21-3d51f54b3707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:38.213147 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:38.256834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 665/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:34:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:38.539843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ea865-a759-4f81-9538-dba868de4d37 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:34:38.541460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0a3ea865-a759-4f81-9538-dba868de4d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:38.552726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:38.644617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ea865-a759-4f81-9538-dba868de4d37 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:34:38.714991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57c6a8-a37f-4879-ae68-a1e2349ed2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:34:38.715589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c6a8-a37f-4879-ae68-a1e2349ed2fe service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:38.715867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c6a8-a37f-4879-ae68-a1e2349ed2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:38.716139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c6a8-a37f-4879-ae68-a1e2349ed2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:38.717731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c57c6a8-a37f-4879-ae68-a1e2349ed2fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:38.777748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:38.822958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57c6a8-a37f-4879-ae68-a1e2349ed2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:34:38.825122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57c6a8-a37f-4879-ae68-a1e2349ed2fe service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:34:38.843817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 652/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:34:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:34:41.784499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2a322-2772-47c4-98ce-a86f811efb67 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:41.785029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2a322-2772-47c4-98ce-a86f811efb67 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:41.785029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2a322-2772-47c4-98ce-a86f811efb67 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:41.786436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 666/1318] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:41.791545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55886e4b-2bef-4d65-92f8-5c8c462311d0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:41.791936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55886e4b-2bef-4d65-92f8-5c8c462311d0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:41.792108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55886e4b-2bef-4d65-92f8-5c8c462311d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:41.857334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-55886e4b-2bef-4d65-92f8-5c8c462311d0 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:41.858568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-55886e4b-2bef-4d65-92f8-5c8c462311d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:41.954738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 653/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:34:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:42.251372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed01ec-df3f-4eb5-866d-753466cb7744 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:42.251650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed01ec-df3f-4eb5-866d-753466cb7744 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:42.251917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed01ec-df3f-4eb5-866d-753466cb7744 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:42.253262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 667/1320] 10.222.0.26 () {54 vars in 964 bytes} [Tue Oct 4 05:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:42.259029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5af25-48e6-434e-8615-b722a0ee6886 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:42.269716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5af25-48e6-434e-8615-b722a0ee6886 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:42.269716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5af25-48e6-434e-8615-b722a0ee6886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:42.281812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:42.335013 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-37d5af25-48e6-434e-8615-b722a0ee6886 None None] MFA Rules not processed for user `3a0feaf83ee14071806f36d01c36b635`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:42.336273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-37d5af25-48e6-434e-8615-b722a0ee6886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:42.410953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 654/1321] 10.222.0.26 () {58 vars in 1061 bytes} [Tue Oct 4 05:34:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:42.419761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed01ec-df3f-4eb5-866d-753466cb7744 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:34:42.421414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-feed01ec-df3f-4eb5-866d-753466cb7744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:42.522408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed01ec-df3f-4eb5-866d-753466cb7744 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:34:42.593799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046941d-bd40-4c64-b498-45a7324387ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:34:42.594668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046941d-bd40-4c64-b498-45a7324387ea service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:42.594959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046941d-bd40-4c64-b498-45a7324387ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:42.595231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046941d-bd40-4c64-b498-45a7324387ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:42.596817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d046941d-bd40-4c64-b498-45a7324387ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:42.604064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:42.703966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046941d-bd40-4c64-b498-45a7324387ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:34:42.705389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046941d-bd40-4c64-b498-45a7324387ea service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:34:42.723591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 668/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:34:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:34:46.223443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4da53-5300-45d1-aadd-b076e5b7880b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:46.223946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4da53-5300-45d1-aadd-b076e5b7880b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:46.223946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4da53-5300-45d1-aadd-b076e5b7880b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:46.225389 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 655/1323] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:46.230380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47991744-d174-4b25-b500-0f1c5822c89f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:46.230777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47991744-d174-4b25-b500-0f1c5822c89f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:46.230922 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47991744-d174-4b25-b500-0f1c5822c89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:46.297071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-47991744-d174-4b25-b500-0f1c5822c89f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:46.298332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-47991744-d174-4b25-b500-0f1c5822c89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:46.362824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:46.404246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 669/1324] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:46.686216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4da53-5300-45d1-aadd-b076e5b7880b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:34:46.687821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-28d4da53-5300-45d1-aadd-b076e5b7880b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:46.791413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4da53-5300-45d1-aadd-b076e5b7880b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:34:46.804506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:46.865758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc32e8-3fc6-4172-a8ae-1badf12175f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:34:46.866381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc32e8-3fc6-4172-a8ae-1badf12175f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:46.866656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc32e8-3fc6-4172-a8ae-1badf12175f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:46.866925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc32e8-3fc6-4172-a8ae-1badf12175f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:46.868499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ebbc32e8-3fc6-4172-a8ae-1badf12175f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:46.979220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc32e8-3fc6-4172-a8ae-1badf12175f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:34:46.980564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc32e8-3fc6-4172-a8ae-1badf12175f8 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:34:46.999990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 656/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:34:49.187877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c91a4-dda6-40a5-9d1b-f279cd64d492 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:49.188399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c91a4-dda6-40a5-9d1b-f279cd64d492 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:49.188399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c91a4-dda6-40a5-9d1b-f279cd64d492 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:49.189716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 670/1326] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:49.195930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfae690-79e1-401d-85a7-af8680c0bbd5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:49.196324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfae690-79e1-401d-85a7-af8680c0bbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:49.196495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfae690-79e1-401d-85a7-af8680c0bbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:49.244067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:49.265737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1bfae690-79e1-401d-85a7-af8680c0bbd5 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:49.267053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1bfae690-79e1-401d-85a7-af8680c0bbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:49.369245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 657/1327] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:49.654436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c91a4-dda6-40a5-9d1b-f279cd64d492 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:34:49.656272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f78c91a4-dda6-40a5-9d1b-f279cd64d492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:49.759688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c91a4-dda6-40a5-9d1b-f279cd64d492 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:34:49.832647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002bf6f-edb2-4f6b-983a-75eaef78b2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:34:49.833190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002bf6f-edb2-4f6b-983a-75eaef78b2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:49.833470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002bf6f-edb2-4f6b-983a-75eaef78b2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:49.833743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002bf6f-edb2-4f6b-983a-75eaef78b2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:49.835394 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0002bf6f-edb2-4f6b-983a-75eaef78b2f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:49.852816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:49.947906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002bf6f-edb2-4f6b-983a-75eaef78b2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:34:49.949347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002bf6f-edb2-4f6b-983a-75eaef78b2f9 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:34:49.967466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 671/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:34:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:34:52.183003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0cf9a-c17f-4a27-b187-4c7c561b0d42 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:52.183512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0cf9a-c17f-4a27-b187-4c7c561b0d42 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:52.183512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0cf9a-c17f-4a27-b187-4c7c561b0d42 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:52.184844 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 658/1329] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:52.190249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd57e7-d582-4fba-af88-a10cc0e0a33d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:52.191205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd57e7-d582-4fba-af88-a10cc0e0a33d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:52.191673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd57e7-d582-4fba-af88-a10cc0e0a33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:52.256231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-97fd57e7-d582-4fba-af88-a10cc0e0a33d None None] MFA Rules not processed for user `f0f33c461e67433593c4ca55f81f4e7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:52.257416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-97fd57e7-d582-4fba-af88-a10cc0e0a33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:52.321120 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:52.353668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 672/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:34:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:52.633912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0cf9a-c17f-4a27-b187-4c7c561b0d42 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:34:52.635596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eab0cf9a-c17f-4a27-b187-4c7c561b0d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:52.749016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0cf9a-c17f-4a27-b187-4c7c561b0d42 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:34:52.778355 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:52.843434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8edbaf-ebe2-4ca9-9195-9412c834ca96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:34:52.844037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8edbaf-ebe2-4ca9-9195-9412c834ca96 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:52.844311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8edbaf-ebe2-4ca9-9195-9412c834ca96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:52.844576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8edbaf-ebe2-4ca9-9195-9412c834ca96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:52.846226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0d8edbaf-ebe2-4ca9-9195-9412c834ca96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:52.959110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8edbaf-ebe2-4ca9-9195-9412c834ca96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:34:52.960533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8edbaf-ebe2-4ca9-9195-9412c834ca96 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:34:53.094400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 659/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:34:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:34:55.651876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90863b-e799-4992-a9be-559c8684b398 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:55.652416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90863b-e799-4992-a9be-559c8684b398 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:55.652416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90863b-e799-4992-a9be-559c8684b398 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:55.653769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 673/1332] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:55.658848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6bfd9-2f37-4576-88a5-d695d248c548 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:55.659249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6bfd9-2f37-4576-88a5-d695d248c548 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:55.659396 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6bfd9-2f37-4576-88a5-d695d248c548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:55.721253 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:55.723761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-24c6bfd9-2f37-4576-88a5-d695d248c548 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:55.724953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-24c6bfd9-2f37-4576-88a5-d695d248c548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:55.819136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 660/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:56.106574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90863b-e799-4992-a9be-559c8684b398 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:34:56.108165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9b90863b-e799-4992-a9be-559c8684b398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:56.208646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90863b-e799-4992-a9be-559c8684b398 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:34:56.279610 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba709694-7a2e-4287-a6dc-3fa2e20c2c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:34:56.280213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba709694-7a2e-4287-a6dc-3fa2e20c2c5c service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:56.280490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba709694-7a2e-4287-a6dc-3fa2e20c2c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:56.280761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba709694-7a2e-4287-a6dc-3fa2e20c2c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:56.282393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ba709694-7a2e-4287-a6dc-3fa2e20c2c5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:56.309506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:56.390637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba709694-7a2e-4287-a6dc-3fa2e20c2c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:34:56.391997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba709694-7a2e-4287-a6dc-3fa2e20c2c5c service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:34:56.409847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 674/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:34:58.603298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f2780-7c44-420b-8805-681029fe9c76 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:58.603825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f2780-7c44-420b-8805-681029fe9c76 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:58.603920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f2780-7c44-420b-8805-681029fe9c76 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:58.605194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 661/1335] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:34:58.610428 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e33e2-9702-4ed3-89aa-1d34c55bc42c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:58.610870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e33e2-9702-4ed3-89aa-1d34c55bc42c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:58.610971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e33e2-9702-4ed3-89aa-1d34c55bc42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:58.674583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7c6e33e2-9702-4ed3-89aa-1d34c55bc42c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:34:58.675675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7c6e33e2-9702-4ed3-89aa-1d34c55bc42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:58.747847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:58.769658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 675/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:34:59.050441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f2780-7c44-420b-8805-681029fe9c76 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:34:59.052000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bf4f2780-7c44-420b-8805-681029fe9c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:59.152767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f2780-7c44-420b-8805-681029fe9c76 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:34:59.181389 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:34:59.225041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fe2f8-15bd-4e7f-87c3-50d3a56f127d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:34:59.225633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fe2f8-15bd-4e7f-87c3-50d3a56f127d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:34:59.225904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fe2f8-15bd-4e7f-87c3-50d3a56f127d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:34:59.226192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fe2f8-15bd-4e7f-87c3-50d3a56f127d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:34:59.228525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b00fe2f8-15bd-4e7f-87c3-50d3a56f127d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:34:59.335742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00fe2f8-15bd-4e7f-87c3-50d3a56f127d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:34:59.337107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00fe2f8-15bd-4e7f-87c3-50d3a56f127d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:34:59.355205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 662/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:02.103561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f9dda-e3ee-4ee6-b7f7-06c57b80d03e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:02.104091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f9dda-e3ee-4ee6-b7f7-06c57b80d03e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:02.104091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f9dda-e3ee-4ee6-b7f7-06c57b80d03e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:02.105468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 676/1338] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:02.110702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed740d1e-78d1-4917-905f-8161f5603460 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:02.111096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed740d1e-78d1-4917-905f-8161f5603460 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:02.111246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed740d1e-78d1-4917-905f-8161f5603460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:02.175434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ed740d1e-78d1-4917-905f-8161f5603460 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:02.176543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed740d1e-78d1-4917-905f-8161f5603460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:02.184500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:02.268798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 663/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:02.547797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f9dda-e3ee-4ee6-b7f7-06c57b80d03e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:02.549415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c20f9dda-e3ee-4ee6-b7f7-06c57b80d03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:02.651604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f9dda-e3ee-4ee6-b7f7-06c57b80d03e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:02.724619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24e226-d0ac-4845-8241-7b2a16fae0db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:02.725226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24e226-d0ac-4845-8241-7b2a16fae0db service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:02.725502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24e226-d0ac-4845-8241-7b2a16fae0db service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:02.725774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24e226-d0ac-4845-8241-7b2a16fae0db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:02.727421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0d24e226-d0ac-4845-8241-7b2a16fae0db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:02.765434 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:02.838839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24e226-d0ac-4845-8241-7b2a16fae0db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:02.840768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24e226-d0ac-4845-8241-7b2a16fae0db service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:02.859376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 677/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:07.117923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494cda5-b75e-4d32-9261-2a0e4b659210 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:07.118522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494cda5-b75e-4d32-9261-2a0e4b659210 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:07.118522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494cda5-b75e-4d32-9261-2a0e4b659210 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:07.119993 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 664/1341] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:07.125103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef7c1c-ba93-40f2-ace5-0f2e8a4ed5a7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:07.125558 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef7c1c-ba93-40f2-ace5-0f2e8a4ed5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:07.125663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef7c1c-ba93-40f2-ace5-0f2e8a4ed5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:07.190529 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-54ef7c1c-ba93-40f2-ace5-0f2e8a4ed5a7 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:07.192002 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-54ef7c1c-ba93-40f2-ace5-0f2e8a4ed5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:07.276283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:07.288807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 678/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:07.575273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494cda5-b75e-4d32-9261-2a0e4b659210 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:07.576825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e494cda5-b75e-4d32-9261-2a0e4b659210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:07.680121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494cda5-b75e-4d32-9261-2a0e4b659210 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:07.718404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:07.756263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6adb0-d4e1-4068-8e30-4633f29da17b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:07.756869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6adb0-d4e1-4068-8e30-4633f29da17b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:07.757146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6adb0-d4e1-4068-8e30-4633f29da17b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:07.757415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6adb0-d4e1-4068-8e30-4633f29da17b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:07.759040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d3f6adb0-d4e1-4068-8e30-4633f29da17b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:07.866487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6adb0-d4e1-4068-8e30-4633f29da17b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:07.867905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6adb0-d4e1-4068-8e30-4633f29da17b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:07.887223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 665/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:35:10.904404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36eb019-f15d-41d4-91bc-5f68fd4d2a92 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:10.904883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36eb019-f15d-41d4-91bc-5f68fd4d2a92 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:10.904991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36eb019-f15d-41d4-91bc-5f68fd4d2a92 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:10.906329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 679/1344] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:35:10.911576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f112-ba93-4325-97b8-cd68ec3c0a05 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:10.911995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f112-ba93-4325-97b8-cd68ec3c0a05 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:10.912104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318f112-ba93-4325-97b8-cd68ec3c0a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:10.977144 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6318f112-ba93-4325-97b8-cd68ec3c0a05 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:10.978394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6318f112-ba93-4325-97b8-cd68ec3c0a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:10.998834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:11.080473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 666/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:11.363234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36eb019-f15d-41d4-91bc-5f68fd4d2a92 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:11.364797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f36eb019-f15d-41d4-91bc-5f68fd4d2a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:11.467516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36eb019-f15d-41d4-91bc-5f68fd4d2a92 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:11.542376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f81ba3-0ac5-4a71-8d48-76d23b6651ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:11.543626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f81ba3-0ac5-4a71-8d48-76d23b6651ba service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:11.543626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f81ba3-0ac5-4a71-8d48-76d23b6651ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:11.543626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f81ba3-0ac5-4a71-8d48-76d23b6651ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:11.545131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b5f81ba3-0ac5-4a71-8d48-76d23b6651ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:11.589587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:11.655672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f81ba3-0ac5-4a71-8d48-76d23b6651ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:11.657104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f81ba3-0ac5-4a71-8d48-76d23b6651ba service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:11.676654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 680/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:15.983857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109d0b-c4b4-47e8-8e29-a7427b2e4719 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:15.984401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109d0b-c4b4-47e8-8e29-a7427b2e4719 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:15.984401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109d0b-c4b4-47e8-8e29-a7427b2e4719 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:15.985753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 667/1347] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:15.991351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841db86e-e14d-456e-baa8-5d86cb0252b5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:15.991856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841db86e-e14d-456e-baa8-5d86cb0252b5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:15.991960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841db86e-e14d-456e-baa8-5d86cb0252b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:16.059663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-841db86e-e14d-456e-baa8-5d86cb0252b5 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:16.060882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-841db86e-e14d-456e-baa8-5d86cb0252b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:16.158871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:16.161143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 681/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:16.442288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109d0b-c4b4-47e8-8e29-a7427b2e4719 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:16.443381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-10109d0b-c4b4-47e8-8e29-a7427b2e4719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:16.551613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109d0b-c4b4-47e8-8e29-a7427b2e4719 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:16.598899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:16.624334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512a815-08f6-4146-966a-20e66389727b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:16.624943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512a815-08f6-4146-966a-20e66389727b service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:16.625219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512a815-08f6-4146-966a-20e66389727b service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:16.625486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512a815-08f6-4146-966a-20e66389727b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:16.627650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b512a815-08f6-4146-966a-20e66389727b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:16.737882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512a815-08f6-4146-966a-20e66389727b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:16.739343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512a815-08f6-4146-966a-20e66389727b service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:16.758683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 668/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:21.632396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae7d56-8858-4674-a128-1408ea626c75 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:21.632396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae7d56-8858-4674-a128-1408ea626c75 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:21.632396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae7d56-8858-4674-a128-1408ea626c75 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:21.638689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666daf65-4a43-409d-ba41-5f46ad6fd265 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:21.639125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666daf65-4a43-409d-ba41-5f46ad6fd265 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:21.639305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666daf65-4a43-409d-ba41-5f46ad6fd265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:21.642605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 682/1350] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:35:21 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:21.735202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-666daf65-4a43-409d-ba41-5f46ad6fd265 None None] MFA Rules not processed for user `e95901cce5d448acb2571976163868a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:21.737395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-666daf65-4a43-409d-ba41-5f46ad6fd265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:21.761564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:21.894191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 669/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:35:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:26.027007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033a887-1265-4db7-a956-81112f9841a7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:26.027995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033a887-1265-4db7-a956-81112f9841a7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:26.028346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033a887-1265-4db7-a956-81112f9841a7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:26.029792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 683/1352] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:26.034799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615441f-34b5-4be1-b6bb-9bedab359f6d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:26.035224 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615441f-34b5-4be1-b6bb-9bedab359f6d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:26.035378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615441f-34b5-4be1-b6bb-9bedab359f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:26.119580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2615441f-34b5-4be1-b6bb-9bedab359f6d None None] MFA Rules not processed for user `e95901cce5d448acb2571976163868a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:26.120917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2615441f-34b5-4be1-b6bb-9bedab359f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:26.197374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:26.234687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 670/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:35:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:27.772254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b3640-32dc-476c-9308-069b9590dc8c None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:27.772773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b3640-32dc-476c-9308-069b9590dc8c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:27.772773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b3640-32dc-476c-9308-069b9590dc8c None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:27.774132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 684/1354] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:27.779132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4e5c2-0b96-4d2c-95c9-63e80999e246 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:27.779528 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4e5c2-0b96-4d2c-95c9-63e80999e246 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:27.779793 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4e5c2-0b96-4d2c-95c9-63e80999e246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:27.855634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-efe4e5c2-0b96-4d2c-95c9-63e80999e246 None None] MFA Rules not processed for user `e95901cce5d448acb2571976163868a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:27.856836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-efe4e5c2-0b96-4d2c-95c9-63e80999e246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:27.965477 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 671/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:35:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:31.476096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca18f95-ebb1-437c-9494-9274ed1be791 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:31.478004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca18f95-ebb1-437c-9494-9274ed1be791 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:31.478487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca18f95-ebb1-437c-9494-9274ed1be791 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:31.479901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 685/1356] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:31.486227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac54e0-daa2-47fe-a6a3-b34182550b63 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:31.487039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac54e0-daa2-47fe-a6a3-b34182550b63 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:31.487315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac54e0-daa2-47fe-a6a3-b34182550b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:31.524252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:31.561050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f3ac54e0-daa2-47fe-a6a3-b34182550b63 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:31.562276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f3ac54e0-daa2-47fe-a6a3-b34182550b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:31.664192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 672/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:35:31.972127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090d1db-7b8c-43bc-9a07-23a3b27fcf10 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:31.972715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090d1db-7b8c-43bc-9a07-23a3b27fcf10 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:31.973562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090d1db-7b8c-43bc-9a07-23a3b27fcf10 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:31.975521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 686/1358] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 05:35:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:31.987493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3738e-3d43-4495-bb39-c7e769ef46fd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:31.987795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3738e-3d43-4495-bb39-c7e769ef46fd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:31.988123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3738e-3d43-4495-bb39-c7e769ef46fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:32.071268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1cf3738e-3d43-4495-bb39-c7e769ef46fd None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:32.072498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1cf3738e-3d43-4495-bb39-c7e769ef46fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:32.129270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:32.198663 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 673/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 05:35:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:32.207086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090d1db-7b8c-43bc-9a07-23a3b27fcf10 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:32.208753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8090d1db-7b8c-43bc-9a07-23a3b27fcf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:32.361788 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090d1db-7b8c-43bc-9a07-23a3b27fcf10 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:32.451204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:32.494886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90badfaf-d0dc-4266-95cd-5131f517aa93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:32.495629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90badfaf-d0dc-4266-95cd-5131f517aa93 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:32.496014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90badfaf-d0dc-4266-95cd-5131f517aa93 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:32.496435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90badfaf-d0dc-4266-95cd-5131f517aa93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:32.498212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-90badfaf-d0dc-4266-95cd-5131f517aa93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:32.617632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90badfaf-d0dc-4266-95cd-5131f517aa93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:32.620202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90badfaf-d0dc-4266-95cd-5131f517aa93 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:32.643527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 687/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:35.038898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7001ea9-e1c6-4c2c-a9b4-f6660d90fcae None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:35.039400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7001ea9-e1c6-4c2c-a9b4-f6660d90fcae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:35.039400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7001ea9-e1c6-4c2c-a9b4-f6660d90fcae None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:35.041027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 674/1361] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:35.049410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b3ef9-b736-4c95-9bf3-cf03c5308c10 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:35.049873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b3ef9-b736-4c95-9bf3-cf03c5308c10 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:35.050101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b3ef9-b736-4c95-9bf3-cf03c5308c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:35.121249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9e9b3ef9-b736-4c95-9bf3-cf03c5308c10 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:35.122584 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9e9b3ef9-b736-4c95-9bf3-cf03c5308c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:35.162003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:35.223084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 688/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:35.515622 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7001ea9-e1c6-4c2c-a9b4-f6660d90fcae None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:35.517420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7001ea9-e1c6-4c2c-a9b4-f6660d90fcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:35.696337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7001ea9-e1c6-4c2c-a9b4-f6660d90fcae None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:35.708059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:35.819812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085e7cb-7c25-4a79-984f-b9919f8cbeee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:35.820426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085e7cb-7c25-4a79-984f-b9919f8cbeee service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:35.820706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085e7cb-7c25-4a79-984f-b9919f8cbeee service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:35.820976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085e7cb-7c25-4a79-984f-b9919f8cbeee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:35.822614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7085e7cb-7c25-4a79-984f-b9919f8cbeee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:35.916272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085e7cb-7c25-4a79-984f-b9919f8cbeee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:35.917797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085e7cb-7c25-4a79-984f-b9919f8cbeee service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:35.936422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 675/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:39.076466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a544c-bc2c-4417-a51c-980e1c1ef9cd None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:39.077424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a544c-bc2c-4417-a51c-980e1c1ef9cd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:39.077699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a544c-bc2c-4417-a51c-980e1c1ef9cd None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:39.079297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 689/1364] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:39.085222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271b3d4-5f4e-45f3-9d53-d030b2019574 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:39.085665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271b3d4-5f4e-45f3-9d53-d030b2019574 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:39.085753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271b3d4-5f4e-45f3-9d53-d030b2019574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:39.102317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:39.161342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0271b3d4-5f4e-45f3-9d53-d030b2019574 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:39.162647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0271b3d4-5f4e-45f3-9d53-d030b2019574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:39.264851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 676/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:39.555242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1955b66-0556-4adf-acfa-f09f0db907ac None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:39.555798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1955b66-0556-4adf-acfa-f09f0db907ac None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:39.556257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1955b66-0556-4adf-acfa-f09f0db907ac None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:39.558015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 690/1366] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 05:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:39.563751 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d8f46-b178-45ab-b09c-1150d9f84934 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:39.564036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d8f46-b178-45ab-b09c-1150d9f84934 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:39.564311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d8f46-b178-45ab-b09c-1150d9f84934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:39.646157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e90d8f46-b178-45ab-b09c-1150d9f84934 None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:39.647406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e90d8f46-b178-45ab-b09c-1150d9f84934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:39.649727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4f6e7-6710-41b2-859c-9cbdd6236952 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:39.650010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4f6e7-6710-41b2-859c-9cbdd6236952 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:39.650311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4f6e7-6710-41b2-859c-9cbdd6236952 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:39.651807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 691/1367] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:39.656877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac5c2d-d053-4c94-bb5a-8b12052f68e4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:39.656877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac5c2d-d053-4c94-bb5a-8b12052f68e4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:39.657168 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac5c2d-d053-4c94-bb5a-8b12052f68e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:39.685880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:39.729110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b4ac5c2d-d053-4c94-bb5a-8b12052f68e4 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:39.730428 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b4ac5c2d-d053-4c94-bb5a-8b12052f68e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:39.779174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 677/1368] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 05:35:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:39.787259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d8f46-b178-45ab-b09c-1150d9f84934 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:39.798794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d8f46-b178-45ab-b09c-1150d9f84934 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:39.842401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 692/1369] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:35:39.933767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d2fbe-b5fd-4ab1-b811-84c1e270f145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:39.934783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d2fbe-b5fd-4ab1-b811-84c1e270f145 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:39.935169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d2fbe-b5fd-4ab1-b811-84c1e270f145 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:39.935543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d2fbe-b5fd-4ab1-b811-84c1e270f145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:39.937256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f05d2fbe-b5fd-4ab1-b811-84c1e270f145 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:39.957658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:40.040187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d2fbe-b5fd-4ab1-b811-84c1e270f145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:40.041780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d2fbe-b5fd-4ab1-b811-84c1e270f145 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:40.061323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 678/1370] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:35:40.174760 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5424ce1-80e2-473b-a5f0-aa6b14aca6c1 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:40.175210 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5424ce1-80e2-473b-a5f0-aa6b14aca6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:40.175317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5424ce1-80e2-473b-a5f0-aa6b14aca6c1 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:40.176647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 693/1371] 10.222.0.26 () {54 vars in 963 bytes} [Tue Oct 4 05:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:40.181848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf2801-f982-4bec-9750-64ff34396874 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:40.182363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf2801-f982-4bec-9750-64ff34396874 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:40.182790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf2801-f982-4bec-9750-64ff34396874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:40.264644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-39bf2801-f982-4bec-9750-64ff34396874 None None] MFA Rules not processed for user `131afaf047d34e88842fffd0d0b76242`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:40.266020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-39bf2801-f982-4bec-9750-64ff34396874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:40.354247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 679/1372] 10.222.0.26 () {58 vars in 1060 bytes} [Tue Oct 4 05:35:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:40.365131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5424ce1-80e2-473b-a5f0-aa6b14aca6c1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:40.367044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d5424ce1-80e2-473b-a5f0-aa6b14aca6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:40.388548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:40.491066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5424ce1-80e2-473b-a5f0-aa6b14aca6c1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:40.572059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fc64b-d0fe-4b40-acd8-e08081d56840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:40.572677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fc64b-d0fe-4b40-acd8-e08081d56840 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:40.572950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fc64b-d0fe-4b40-acd8-e08081d56840 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:40.573220 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fc64b-d0fe-4b40-acd8-e08081d56840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:40.574848 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3b6fc64b-d0fe-4b40-acd8-e08081d56840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:40.650873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:40.690274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fc64b-d0fe-4b40-acd8-e08081d56840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:40.691985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fc64b-d0fe-4b40-acd8-e08081d56840 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:40.710955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 694/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:42.487138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0807fca-1035-4c62-9669-a03debc85c03 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:42.487682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0807fca-1035-4c62-9669-a03debc85c03 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:42.487682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0807fca-1035-4c62-9669-a03debc85c03 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:42.489045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 680/1374] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:42.494469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e067df9-3c3a-4585-b5d6-03aaf8066678 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:42.494908 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e067df9-3c3a-4585-b5d6-03aaf8066678 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:42.495014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e067df9-3c3a-4585-b5d6-03aaf8066678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:42.563746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6e067df9-3c3a-4585-b5d6-03aaf8066678 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:42.565243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6e067df9-3c3a-4585-b5d6-03aaf8066678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:42.667555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 695/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:42.959725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0807fca-1035-4c62-9669-a03debc85c03 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:42.961275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d0807fca-1035-4c62-9669-a03debc85c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:42.969611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:43.115586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0807fca-1035-4c62-9669-a03debc85c03 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:43.203312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:43.237547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c2405-77ff-4c42-b6f4-c17297d75278 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:43.238248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c2405-77ff-4c42-b6f4-c17297d75278 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:43.238536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c2405-77ff-4c42-b6f4-c17297d75278 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:43.238807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c2405-77ff-4c42-b6f4-c17297d75278 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:43.240423 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c46c2405-77ff-4c42-b6f4-c17297d75278 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:43.337680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c2405-77ff-4c42-b6f4-c17297d75278 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:43.339309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c2405-77ff-4c42-b6f4-c17297d75278 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:43.358294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 681/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:45.712322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7711457-6c3d-4741-b47e-557128d66005 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:45.712878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7711457-6c3d-4741-b47e-557128d66005 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:45.712878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7711457-6c3d-4741-b47e-557128d66005 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:45.714378 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 696/1377] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:45.719409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca079a05-4313-416b-b6c4-9f945592a687 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:45.719799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca079a05-4313-416b-b6c4-9f945592a687 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:45.719981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca079a05-4313-416b-b6c4-9f945592a687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:45.787721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ca079a05-4313-416b-b6c4-9f945592a687 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:45.788929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca079a05-4313-416b-b6c4-9f945592a687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:45.835045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:45.895836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 682/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:46.184328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7711457-6c3d-4741-b47e-557128d66005 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:46.185916 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c7711457-6c3d-4741-b47e-557128d66005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:46.221588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:46.343159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7711457-6c3d-4741-b47e-557128d66005 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:46.461129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:46.473894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59989b2e-68bd-496c-b979-28c47b103371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:46.475876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59989b2e-68bd-496c-b979-28c47b103371 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:46.475876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59989b2e-68bd-496c-b979-28c47b103371 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:46.475876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59989b2e-68bd-496c-b979-28c47b103371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:46.476757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-59989b2e-68bd-496c-b979-28c47b103371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:46.571682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59989b2e-68bd-496c-b979-28c47b103371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:46.573431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59989b2e-68bd-496c-b979-28c47b103371 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:46.592195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 697/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:48.917533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae75c9-e320-4542-9b46-5bcb7fc62cc6 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:48.918083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae75c9-e320-4542-9b46-5bcb7fc62cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:48.918364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae75c9-e320-4542-9b46-5bcb7fc62cc6 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:48.920160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 683/1380] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:48.927353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef227-55bc-4376-98fb-a86f50487c58 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:48.927866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef227-55bc-4376-98fb-a86f50487c58 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:48.928035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef227-55bc-4376-98fb-a86f50487c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:48.997810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-566ef227-55bc-4376-98fb-a86f50487c58 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:48.999479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-566ef227-55bc-4376-98fb-a86f50487c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:49.074985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:49.121626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 698/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:49.493184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae75c9-e320-4542-9b46-5bcb7fc62cc6 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:49.494764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-19ae75c9-e320-4542-9b46-5bcb7fc62cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:49.647436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae75c9-e320-4542-9b46-5bcb7fc62cc6 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:49.674215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:49.762776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500d432-d959-4d6a-b127-6441ae5b561c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:49.763303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500d432-d959-4d6a-b127-6441ae5b561c service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:49.763583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500d432-d959-4d6a-b127-6441ae5b561c service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:49.763852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500d432-d959-4d6a-b127-6441ae5b561c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:49.765468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2500d432-d959-4d6a-b127-6441ae5b561c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:49.859895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500d432-d959-4d6a-b127-6441ae5b561c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:49.861631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500d432-d959-4d6a-b127-6441ae5b561c service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:49.879696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 684/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:52.203890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728bcc2-9b45-4b73-a923-d8f178abbbdb None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:52.204425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728bcc2-9b45-4b73-a923-d8f178abbbdb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:52.204425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728bcc2-9b45-4b73-a923-d8f178abbbdb None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:52.205930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 699/1383] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:52.211277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4b67-48a7-4a83-9b5f-062cf0acbca3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:52.211680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4b67-48a7-4a83-9b5f-062cf0acbca3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:52.211812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4b67-48a7-4a83-9b5f-062cf0acbca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:52.251754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:52.276931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-05ee4b67-48a7-4a83-9b5f-062cf0acbca3 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:52.278179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-05ee4b67-48a7-4a83-9b5f-062cf0acbca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:52.378979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 685/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:52.669931 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728bcc2-9b45-4b73-a923-d8f178abbbdb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:52.671585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6728bcc2-9b45-4b73-a923-d8f178abbbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:52.838063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728bcc2-9b45-4b73-a923-d8f178abbbdb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:52.898379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:52.968067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9a6c8-316d-4496-842f-57f1b8e64c8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:52.969284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9a6c8-316d-4496-842f-57f1b8e64c8f service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:52.969715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9a6c8-316d-4496-842f-57f1b8e64c8f service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:52.970070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9a6c8-316d-4496-842f-57f1b8e64c8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:52.972089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2ac9a6c8-316d-4496-842f-57f1b8e64c8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:53.081994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9a6c8-316d-4496-842f-57f1b8e64c8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:53.083755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9a6c8-316d-4496-842f-57f1b8e64c8f service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:53.102721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 700/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:55.249113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d279c1f-9d54-43a8-88c7-a4af6edc3076 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:55.249660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d279c1f-9d54-43a8-88c7-a4af6edc3076 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:55.249660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d279c1f-9d54-43a8-88c7-a4af6edc3076 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:55.250995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 686/1386] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:55.256343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ebfff-2230-4f03-9818-d02696e2e1d2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:55.256763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ebfff-2230-4f03-9818-d02696e2e1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:55.256894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ebfff-2230-4f03-9818-d02696e2e1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:55.322823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c17ebfff-2230-4f03-9818-d02696e2e1d2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:55.324032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c17ebfff-2230-4f03-9818-d02696e2e1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:55.326024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:55.419620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 701/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:55.708254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d279c1f-9d54-43a8-88c7-a4af6edc3076 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:55.709795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3d279c1f-9d54-43a8-88c7-a4af6edc3076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:55.813691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:55.855254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d279c1f-9d54-43a8-88c7-a4af6edc3076 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:55.970922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1224682-b42b-458a-8445-e598d807421b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:55.971445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1224682-b42b-458a-8445-e598d807421b service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:55.971721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1224682-b42b-458a-8445-e598d807421b service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:55.971990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1224682-b42b-458a-8445-e598d807421b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:55.973635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f1224682-b42b-458a-8445-e598d807421b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:56.049789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:56.066925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1224682-b42b-458a-8445-e598d807421b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:56.068563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1224682-b42b-458a-8445-e598d807421b service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:56.087221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 687/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:56.141071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d52006-7d1f-43fe-98e0-e2283dc5ef7a None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:56.141357 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d52006-7d1f-43fe-98e0-e2283dc5ef7a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:56.141634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d52006-7d1f-43fe-98e0-e2283dc5ef7a None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:56.143044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 702/1389] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:56.148184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4b9a2-9ebd-411a-b110-7580ffb8473e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:56.148476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4b9a2-9ebd-411a-b110-7580ffb8473e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:56.148748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4b9a2-9ebd-411a-b110-7580ffb8473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:56.216387 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a7e4b9a2-9ebd-411a-b110-7580ffb8473e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:56.221434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a7e4b9a2-9ebd-411a-b110-7580ffb8473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:56.324093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 688/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:56.627307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14552bd-f8ab-44a2-b94d-627cfd708154 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:56.627799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14552bd-f8ab-44a2-b94d-627cfd708154 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:56.627917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14552bd-f8ab-44a2-b94d-627cfd708154 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:56.629188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 703/1391] 10.222.0.26 () {54 vars in 962 bytes} [Tue Oct 4 05:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:56.634586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf8035-d077-419a-9070-a56efd97621f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:56.634945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf8035-d077-419a-9070-a56efd97621f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:56.635244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf8035-d077-419a-9070-a56efd97621f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:56.698398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:56.717227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-edcf8035-d077-419a-9070-a56efd97621f None None] MFA Rules not processed for user `c34eb8fcef0543c282b3887dfeb74c15`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:56.719016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-edcf8035-d077-419a-9070-a56efd97621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:56.794662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 689/1392] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Oct 4 05:35:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:56.803726 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14552bd-f8ab-44a2-b94d-627cfd708154 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:56.805303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e14552bd-f8ab-44a2-b94d-627cfd708154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:56.912561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14552bd-f8ab-44a2-b94d-627cfd708154 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:56.945012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:56.986832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59b343-78dc-4765-a582-812df2719cc2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34eb8fcef0543c282b3887dfeb74c15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:56.987564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59b343-78dc-4765-a582-812df2719cc2 service glance] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:56.987844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59b343-78dc-4765-a582-812df2719cc2 service glance] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:56.988118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59b343-78dc-4765-a582-812df2719cc2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:56.989692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3f59b343-78dc-4765-a582-812df2719cc2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:57.096155 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59b343-78dc-4765-a582-812df2719cc2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:57.097586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59b343-78dc-4765-a582-812df2719cc2 service glance] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:57.114968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 704/1393] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:35:58.059112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e5275-3910-48cb-980c-116a79ad8071 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:58.059656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e5275-3910-48cb-980c-116a79ad8071 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:58.059656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e5275-3910-48cb-980c-116a79ad8071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:58.128955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2c0e5275-3910-48cb-980c-116a79ad8071 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:58.130033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c0e5275-3910-48cb-980c-116a79ad8071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:58.214840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 690/1394] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:35:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:58.257600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6984e7-f83c-40a2-a714-9dd237950b5b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:58.258108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6984e7-f83c-40a2-a714-9dd237950b5b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:58.258248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6984e7-f83c-40a2-a714-9dd237950b5b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:58.259519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 705/1395] 10.222.0.26 () {54 vars in 943 bytes} [Tue Oct 4 05:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:35:58.266368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d24f7e-8e2b-4510-b07b-463d4ab94088 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:58.266653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d24f7e-8e2b-4510-b07b-463d4ab94088 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:58.266926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d24f7e-8e2b-4510-b07b-463d4ab94088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:58.277140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:58.353247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b7d24f7e-8e2b-4510-b07b-463d4ab94088 None None] MFA Rules not processed for user `783ad0cd9721409e8a7114cecef16aab`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:58.354496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b7d24f7e-8e2b-4510-b07b-463d4ab94088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:58.424842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b7994-12b5-4eba-b4e1-6795b64333df None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:58.425038 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b7994-12b5-4eba-b4e1-6795b64333df None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:58.425424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b7994-12b5-4eba-b4e1-6795b64333df None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:58.426940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 706/1396] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:35:58.432010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0598a-218b-48e9-b671-d37bcaffd98b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:58.432400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0598a-218b-48e9-b671-d37bcaffd98b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:58.432565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0598a-218b-48e9-b671-d37bcaffd98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:58.462226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 691/1397] 10.222.0.26 () {58 vars in 1040 bytes} [Tue Oct 4 05:35:58 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:35:58.471623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d24f7e-8e2b-4510-b07b-463d4ab94088 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:58.481157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d24f7e-8e2b-4510-b07b-463d4ab94088 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:58.501674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e3b0598a-218b-48e9-b671-d37bcaffd98b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:35:58.502927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e3b0598a-218b-48e9-b671-d37bcaffd98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:58.516635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:58.539041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:58.586580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771f04c-de0b-4906-908f-78cb40f26ac0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:58.587184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771f04c-de0b-4906-908f-78cb40f26ac0 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:58.587460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771f04c-de0b-4906-908f-78cb40f26ac0 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:58.587730 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771f04c-de0b-4906-908f-78cb40f26ac0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:58.589372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4771f04c-de0b-4906-908f-78cb40f26ac0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:58.602082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 707/1398] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:35:58.673982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771f04c-de0b-4906-908f-78cb40f26ac0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:58.675684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771f04c-de0b-4906-908f-78cb40f26ac0 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:58.683761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 692/1399] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:35:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 05:35:58.895911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0598a-218b-48e9-b671-d37bcaffd98b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:35:58.898336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e3b0598a-218b-48e9-b671-d37bcaffd98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:59.133182 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0598a-218b-48e9-b671-d37bcaffd98b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:35:59.163712 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:35:59.264568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1194c6a-86c4-4def-8801-47e71126f857 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:35:59.265243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194c6a-86c4-4def-8801-47e71126f857 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:35:59.265511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194c6a-86c4-4def-8801-47e71126f857 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:35:59.265867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194c6a-86c4-4def-8801-47e71126f857 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:35:59.267547 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f1194c6a-86c4-4def-8801-47e71126f857 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:35:59.376000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1194c6a-86c4-4def-8801-47e71126f857 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:35:59.377639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1194c6a-86c4-4def-8801-47e71126f857 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:35:59.398164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 708/1400] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:01.760415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1466de3-6306-47d8-8fed-58eb5b07e002 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:01.761513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1466de3-6306-47d8-8fed-58eb5b07e002 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:01.762040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1466de3-6306-47d8-8fed-58eb5b07e002 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:01.763903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 693/1401] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:36:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:01.769522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff22062-d313-42d0-8181-9582da050e53 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:01.769939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff22062-d313-42d0-8181-9582da050e53 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:01.770185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff22062-d313-42d0-8181-9582da050e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:01.817929 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:01.842359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-aff22062-d313-42d0-8181-9582da050e53 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:01.844205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aff22062-d313-42d0-8181-9582da050e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:01.973349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 709/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:02.267754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1466de3-6306-47d8-8fed-58eb5b07e002 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:02.269453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d1466de3-6306-47d8-8fed-58eb5b07e002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:02.367923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:02.432909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1466de3-6306-47d8-8fed-58eb5b07e002 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:02.578701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6eed2-3518-4962-a3d9-2f9837c042d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:02.579317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6eed2-3518-4962-a3d9-2f9837c042d1 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:02.579592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6eed2-3518-4962-a3d9-2f9837c042d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:02.579780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6eed2-3518-4962-a3d9-2f9837c042d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:02.581410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b6a6eed2-3518-4962-a3d9-2f9837c042d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:02.661871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:02.701483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6eed2-3518-4962-a3d9-2f9837c042d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:02.703181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6eed2-3518-4962-a3d9-2f9837c042d1 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:02.725108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 694/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:05.596878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47b2bf-51f4-42a1-90f8-a6b74509db86 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:05.597475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47b2bf-51f4-42a1-90f8-a6b74509db86 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:05.597475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47b2bf-51f4-42a1-90f8-a6b74509db86 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:05.602586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 710/1404] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:36:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:05.604173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7cc9b-bdc7-4969-a481-1f3f9333d5a1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:05.604630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7cc9b-bdc7-4969-a481-1f3f9333d5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:05.604716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7cc9b-bdc7-4969-a481-1f3f9333d5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:05.677966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8bd7cc9b-bdc7-4969-a481-1f3f9333d5a1 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:05.679320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8bd7cc9b-bdc7-4969-a481-1f3f9333d5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:05.785896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 695/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:06.083815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47b2bf-51f4-42a1-90f8-a6b74509db86 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:06.085350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6f47b2bf-51f4-42a1-90f8-a6b74509db86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:06.206108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:06.251756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47b2bf-51f4-42a1-90f8-a6b74509db86 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:06.374852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37467373-b294-4341-8a9a-778120e202bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:06.375467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37467373-b294-4341-8a9a-778120e202bf service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:06.375774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37467373-b294-4341-8a9a-778120e202bf service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:06.376017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37467373-b294-4341-8a9a-778120e202bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:06.377621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-37467373-b294-4341-8a9a-778120e202bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:06.456248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:06.477310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37467373-b294-4341-8a9a-778120e202bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:06.478952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37467373-b294-4341-8a9a-778120e202bf service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:06.501680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 711/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:08.899714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a614ed-dad4-4270-9fcf-6a11bcd42415 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:08.900174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a614ed-dad4-4270-9fcf-6a11bcd42415 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:08.900262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a614ed-dad4-4270-9fcf-6a11bcd42415 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:08.901966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 696/1407] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:08.907068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267729f-708e-406f-a3fe-ccc0b679af9b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:08.907472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267729f-708e-406f-a3fe-ccc0b679af9b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:08.907659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267729f-708e-406f-a3fe-ccc0b679af9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:08.977953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d267729f-708e-406f-a3fe-ccc0b679af9b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:08.979209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d267729f-708e-406f-a3fe-ccc0b679af9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:09.098436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 712/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:09.383056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a614ed-dad4-4270-9fcf-6a11bcd42415 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:09.384703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-38a614ed-dad4-4270-9fcf-6a11bcd42415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:09.425897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:09.547255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a614ed-dad4-4270-9fcf-6a11bcd42415 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:09.667305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:09.674813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563e9b8-a62b-4ad2-ad6e-d064c2c964c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:09.679348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563e9b8-a62b-4ad2-ad6e-d064c2c964c6 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:09.679348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563e9b8-a62b-4ad2-ad6e-d064c2c964c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:09.679348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563e9b8-a62b-4ad2-ad6e-d064c2c964c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:09.679348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7563e9b8-a62b-4ad2-ad6e-d064c2c964c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:09.777029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563e9b8-a62b-4ad2-ad6e-d064c2c964c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:09.778659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563e9b8-a62b-4ad2-ad6e-d064c2c964c6 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:09.801817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 697/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:19.091399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=77 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:19.795271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=48 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:22.809517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef6c8-fc3a-4e8a-9219-1267b8e3076b None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:22.810049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef6c8-fc3a-4e8a-9219-1267b8e3076b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:22.810049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef6c8-fc3a-4e8a-9219-1267b8e3076b None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:22.811385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 713/1410] 10.222.0.26 () {54 vars in 918 bytes} [Tue Oct 4 05:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:22.816781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e48c07-f70f-4618-ba4e-83b4d4602eb9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:22.817256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e48c07-f70f-4618-ba4e-83b4d4602eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:22.817362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e48c07-f70f-4618-ba4e-83b4d4602eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:22.893048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d2e48c07-f70f-4618-ba4e-83b4d4602eb9 None None] MFA Rules not processed for user `e95901cce5d448acb2571976163868a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:22.894340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d2e48c07-f70f-4618-ba4e-83b4d4602eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:22.996863 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 698/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Oct 4 05:36:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:36:23.010841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e529b-56e5-45c7-b780-6d04549ba428 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:23.011117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e529b-56e5-45c7-b780-6d04549ba428 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:23.011387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e529b-56e5-45c7-b780-6d04549ba428 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:23.012694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 714/1412] 10.222.0.26 () {54 vars in 918 bytes} [Tue Oct 4 05:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:23.018034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0831-2515-4b59-90fe-b01755509fd1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:23.018347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0831-2515-4b59-90fe-b01755509fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:23.018623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e0831-2515-4b59-90fe-b01755509fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:23.090299 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:23.093282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-711e0831-2515-4b59-90fe-b01755509fd1 None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:23.094526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-711e0831-2515-4b59-90fe-b01755509fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:23.208830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 699/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Oct 4 05:36:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:25.802091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb644d7-2299-4bd6-8460-f35b22d5e4f8 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:25.802659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb644d7-2299-4bd6-8460-f35b22d5e4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:25.802659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb644d7-2299-4bd6-8460-f35b22d5e4f8 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:25.803988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:25.809387 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae16fb6-5c7f-4d52-9c7a-6f35db2f31ad None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:25.809818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae16fb6-5c7f-4d52-9c7a-6f35db2f31ad None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:25.809928 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae16fb6-5c7f-4d52-9c7a-6f35db2f31ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:25.877470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dae16fb6-5c7f-4d52-9c7a-6f35db2f31ad None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:25.878716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dae16fb6-5c7f-4d52-9c7a-6f35db2f31ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:25.933230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:25.975596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:26.265340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb644d7-2299-4bd6-8460-f35b22d5e4f8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:26.266918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dbb644d7-2299-4bd6-8460-f35b22d5e4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:26.410995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb644d7-2299-4bd6-8460-f35b22d5e4f8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:26.484187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:26.523779 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe08f7-817e-4a99-9edc-9ba00dcb9d41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:26.524410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe08f7-817e-4a99-9edc-9ba00dcb9d41 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:26.524688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe08f7-817e-4a99-9edc-9ba00dcb9d41 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:26.524957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe08f7-817e-4a99-9edc-9ba00dcb9d41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:26.526623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9ffe08f7-817e-4a99-9edc-9ba00dcb9d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:26.614986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe08f7-817e-4a99-9edc-9ba00dcb9d41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:26.616566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe08f7-817e-4a99-9edc-9ba00dcb9d41 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:26.633994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 716/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:27.776639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1bcc1-b433-4704-9f43-ac0a49e6a23b None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:27.777188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1bcc1-b433-4704-9f43-ac0a49e6a23b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:27.777188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1bcc1-b433-4704-9f43-ac0a49e6a23b None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:27.778499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:27.785175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378cd38-853f-4310-9961-c1a4a133d4db None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:27.785644 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378cd38-853f-4310-9961-c1a4a133d4db None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:27.785744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378cd38-853f-4310-9961-c1a4a133d4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:27.859939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1378cd38-853f-4310-9961-c1a4a133d4db None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:27.861137 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1378cd38-853f-4310-9961-c1a4a133d4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:27.887798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:27.958387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:28.112129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1bcc1-b433-4704-9f43-ac0a49e6a23b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:28.113705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-66f1bcc1-b433-4704-9f43-ac0a49e6a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:28.253646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1bcc1-b433-4704-9f43-ac0a49e6a23b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:28.293017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:28.364536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5e3f7-b5ed-40e3-a0ed-19a27735391f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:28.365064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5e3f7-b5ed-40e3-a0ed-19a27735391f service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:28.365340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5e3f7-b5ed-40e3-a0ed-19a27735391f service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:28.365606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5e3f7-b5ed-40e3-a0ed-19a27735391f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:28.367235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eaf5e3f7-b5ed-40e3-a0ed-19a27735391f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:28.455327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5e3f7-b5ed-40e3-a0ed-19a27735391f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:28.457103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5e3f7-b5ed-40e3-a0ed-19a27735391f service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:28.475090 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 702/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:29.597039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4476b-5715-49e5-a63d-288b6f8948ce None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:29.597577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4476b-5715-49e5-a63d-288b6f8948ce None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:29.597577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4476b-5715-49e5-a63d-288b6f8948ce None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:29.598959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:29.606402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f99062-b8d9-4e56-9ba4-4799c0f860a1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:29.606832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f99062-b8d9-4e56-9ba4-4799c0f860a1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:29.606981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f99062-b8d9-4e56-9ba4-4799c0f860a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:29.660684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:29.681474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-88f99062-b8d9-4e56-9ba4-4799c0f860a1 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:29.682714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-88f99062-b8d9-4e56-9ba4-4799c0f860a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:29.776937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:29.928591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4476b-5715-49e5-a63d-288b6f8948ce None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:29.930124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-87e4476b-5715-49e5-a63d-288b6f8948ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:30.073254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4476b-5715-49e5-a63d-288b6f8948ce None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:30.085702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:30.184438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787498b-2c21-4ca4-bc57-6508bb879065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:30.184961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787498b-2c21-4ca4-bc57-6508bb879065 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:30.185754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787498b-2c21-4ca4-bc57-6508bb879065 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:30.186025 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787498b-2c21-4ca4-bc57-6508bb879065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:30.187621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c787498b-2c21-4ca4-bc57-6508bb879065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:30.274097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787498b-2c21-4ca4-bc57-6508bb879065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:30.275719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787498b-2c21-4ca4-bc57-6508bb879065 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:30.293190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 719/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:31.405590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c421583-6326-4394-89bd-372c7dd42a24 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:31.406159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c421583-6326-4394-89bd-372c7dd42a24 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:31.406159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c421583-6326-4394-89bd-372c7dd42a24 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:31.407593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:31.413862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801257e-54bf-4f41-9e20-850bf6abc60e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:31.414325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801257e-54bf-4f41-9e20-850bf6abc60e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:31.414460 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801257e-54bf-4f41-9e20-850bf6abc60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:31.434078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:31.489154 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9801257e-54bf-4f41-9e20-850bf6abc60e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:31.490398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9801257e-54bf-4f41-9e20-850bf6abc60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:31.584887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:31.733999 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c421583-6326-4394-89bd-372c7dd42a24 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:31.735473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9c421583-6326-4394-89bd-372c7dd42a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:31.842901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:31.880161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c421583-6326-4394-89bd-372c7dd42a24 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:31.993285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aced5-9c36-43c4-9288-114649a704c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:31.993815 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aced5-9c36-43c4-9288-114649a704c6 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:31.994091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aced5-9c36-43c4-9288-114649a704c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:31.994400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aced5-9c36-43c4-9288-114649a704c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:31.996011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c94aced5-9c36-43c4-9288-114649a704c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:32.072318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:32.085696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aced5-9c36-43c4-9288-114649a704c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:32.087309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aced5-9c36-43c4-9288-114649a704c6 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:32.105835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 705/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:33.219775 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b106b-7b74-4a0e-98e0-b10554c0b6be None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:33.220343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b106b-7b74-4a0e-98e0-b10554c0b6be None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:33.220343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b106b-7b74-4a0e-98e0-b10554c0b6be None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:33.221662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:33.227939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d42982-eec1-4288-9eee-85d2b6c93ae7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:33.228376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d42982-eec1-4288-9eee-85d2b6c93ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:33.228487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d42982-eec1-4288-9eee-85d2b6c93ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:33.303875 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-98d42982-eec1-4288-9eee-85d2b6c93ae7 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:33.305075 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-98d42982-eec1-4288-9eee-85d2b6c93ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:33.399094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:33.549944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b106b-7b74-4a0e-98e0-b10554c0b6be None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:33.551415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d62b106b-7b74-4a0e-98e0-b10554c0b6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:33.634761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:33.694892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b106b-7b74-4a0e-98e0-b10554c0b6be None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:33.804416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bb690-74cb-4766-b85b-8f02d09f5a94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:33.804948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bb690-74cb-4766-b85b-8f02d09f5a94 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:33.805227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bb690-74cb-4766-b85b-8f02d09f5a94 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:33.805495 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bb690-74cb-4766-b85b-8f02d09f5a94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:33.807104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-537bb690-74cb-4766-b85b-8f02d09f5a94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:33.854857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:33.894900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537bb690-74cb-4766-b85b-8f02d09f5a94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:33.896499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537bb690-74cb-4766-b85b-8f02d09f5a94 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:33.914409 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 722/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:35.033626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc5255-3f43-42e4-a16c-07f9dacac515 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:35.034276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc5255-3f43-42e4-a16c-07f9dacac515 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:35.034276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc5255-3f43-42e4-a16c-07f9dacac515 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:35.035546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:35.041753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceb7d3-d416-4c2c-a2f8-e3bafebe55d6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:35.042199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceb7d3-d416-4c2c-a2f8-e3bafebe55d6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:35.042350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceb7d3-d416-4c2c-a2f8-e3bafebe55d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:35.118413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-81ceb7d3-d416-4c2c-a2f8-e3bafebe55d6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:35.119624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-81ceb7d3-d416-4c2c-a2f8-e3bafebe55d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:35.215758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:35.367613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc5255-3f43-42e4-a16c-07f9dacac515 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:35.369169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ddcc5255-3f43-42e4-a16c-07f9dacac515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:35.417237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:35.512298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc5255-3f43-42e4-a16c-07f9dacac515 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:35.620978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf2f5c-59a4-4ca7-96b5-591a843552cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:35.621582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf2f5c-59a4-4ca7-96b5-591a843552cf service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:35.621856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf2f5c-59a4-4ca7-96b5-591a843552cf service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:35.622121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf2f5c-59a4-4ca7-96b5-591a843552cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:35.623766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b4cf2f5c-59a4-4ca7-96b5-591a843552cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:35.630578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:35.712785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf2f5c-59a4-4ca7-96b5-591a843552cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:35.714390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf2f5c-59a4-4ca7-96b5-591a843552cf service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:35.732517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 708/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:36.830496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17ca39-56f8-4475-a3b5-c48dae1aa6e7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:36.831044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17ca39-56f8-4475-a3b5-c48dae1aa6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:36.831044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17ca39-56f8-4475-a3b5-c48dae1aa6e7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:36.832336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:36.838466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f957b8c-4577-4f33-8a5a-044549d500bd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:36.838938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f957b8c-4577-4f33-8a5a-044549d500bd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:36.839137 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f957b8c-4577-4f33-8a5a-044549d500bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:36.914947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0f957b8c-4577-4f33-8a5a-044549d500bd None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:36.916152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0f957b8c-4577-4f33-8a5a-044549d500bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:36.982539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:37.010158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:37.158626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17ca39-56f8-4475-a3b5-c48dae1aa6e7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:37.160130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3b17ca39-56f8-4475-a3b5-c48dae1aa6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:37.177401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:37.302765 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17ca39-56f8-4475-a3b5-c48dae1aa6e7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:37.387610 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:37.413556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9eeff9-8b3a-4f4a-bd63-fb3d8ef4a6d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:37.414102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9eeff9-8b3a-4f4a-bd63-fb3d8ef4a6d5 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:37.414439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9eeff9-8b3a-4f4a-bd63-fb3d8ef4a6d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:37.414714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9eeff9-8b3a-4f4a-bd63-fb3d8ef4a6d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:37.416458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ff9eeff9-8b3a-4f4a-bd63-fb3d8ef4a6d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:37.508141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9eeff9-8b3a-4f4a-bd63-fb3d8ef4a6d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:37.509866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9eeff9-8b3a-4f4a-bd63-fb3d8ef4a6d5 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:37.528763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 725/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:38.612897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d870e-fd55-4723-994d-5469305c3bdf None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:38.613473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d870e-fd55-4723-994d-5469305c3bdf None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:38.613473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d870e-fd55-4723-994d-5469305c3bdf None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:38.614832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:38.620934 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d2614-ad0d-41fb-a0bf-8dfb62b5cf05 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:38.621460 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d2614-ad0d-41fb-a0bf-8dfb62b5cf05 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:38.621651 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d2614-ad0d-41fb-a0bf-8dfb62b5cf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:38.697448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e68d2614-ad0d-41fb-a0bf-8dfb62b5cf05 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:38.698691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e68d2614-ad0d-41fb-a0bf-8dfb62b5cf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:38.739744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:38.792913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:38.947648 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d870e-fd55-4723-994d-5469305c3bdf None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:38.949172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-582d870e-fd55-4723-994d-5469305c3bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:39.092388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d870e-fd55-4723-994d-5469305c3bdf None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:39.145849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:39.202976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2237f-6e13-4380-b964-dc450ad81c20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:39.203575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2237f-6e13-4380-b964-dc450ad81c20 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:39.203853 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2237f-6e13-4380-b964-dc450ad81c20 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:39.204123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2237f-6e13-4380-b964-dc450ad81c20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:39.205812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7dd2237f-6e13-4380-b964-dc450ad81c20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:39.292572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2237f-6e13-4380-b964-dc450ad81c20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:39.294772 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2237f-6e13-4380-b964-dc450ad81c20 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:39.312870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 711/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:40.408026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfd215-f59f-432b-8815-a1f52e4c6be7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:40.408579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfd215-f59f-432b-8815-a1f52e4c6be7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:40.408579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfd215-f59f-432b-8815-a1f52e4c6be7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:40.409919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:40.416246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc669cd-9418-4ea4-8c8d-51e3ef4e789e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:40.425739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc669cd-9418-4ea4-8c8d-51e3ef4e789e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:40.425739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc669cd-9418-4ea4-8c8d-51e3ef4e789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:40.492436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8bc669cd-9418-4ea4-8c8d-51e3ef4e789e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:40.493650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8bc669cd-9418-4ea4-8c8d-51e3ef4e789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:40.495650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:40.589553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:40.739091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfd215-f59f-432b-8815-a1f52e4c6be7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:40.740521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-87bfd215-f59f-432b-8815-a1f52e4c6be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:40.881503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfd215-f59f-432b-8815-a1f52e4c6be7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:40.908156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:40.993228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6818c-50d4-47f0-84f2-c698387d3e7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:40.993755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6818c-50d4-47f0-84f2-c698387d3e7e service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:40.994027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6818c-50d4-47f0-84f2-c698387d3e7e service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:40.994323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6818c-50d4-47f0-84f2-c698387d3e7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:40.995895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a2d6818c-50d4-47f0-84f2-c698387d3e7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:41.084268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6818c-50d4-47f0-84f2-c698387d3e7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:41.085841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6818c-50d4-47f0-84f2-c698387d3e7e service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:41.103569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 728/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:43.180624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20f4db-7743-4644-8eaf-857d78252741 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:43.181183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20f4db-7743-4644-8eaf-857d78252741 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:43.181183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20f4db-7743-4644-8eaf-857d78252741 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:43.182540 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:43.188574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c4f9-c587-402a-adb2-0a6fdea95abe None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:43.189012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c4f9-c587-402a-adb2-0a6fdea95abe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:43.189152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c4f9-c587-402a-adb2-0a6fdea95abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:43.229736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:43.255701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e8d4c4f9-c587-402a-adb2-0a6fdea95abe None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:43.256900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e8d4c4f9-c587-402a-adb2-0a6fdea95abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:43.350938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:43.632313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20f4db-7743-4644-8eaf-857d78252741 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:43.633844 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ea20f4db-7743-4644-8eaf-857d78252741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:43.735687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20f4db-7743-4644-8eaf-857d78252741 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:43.766944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:43.806642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162b645-294b-484d-8937-e092b39862f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:43.807234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162b645-294b-484d-8937-e092b39862f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:43.807507 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162b645-294b-484d-8937-e092b39862f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:43.807774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162b645-294b-484d-8937-e092b39862f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:43.809340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f162b645-294b-484d-8937-e092b39862f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:43.918384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162b645-294b-484d-8937-e092b39862f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:43.919805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162b645-294b-484d-8937-e092b39862f4 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:43.939075 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 714/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:46.232915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f1842-4bc6-4fb1-9b8e-bf558ccf72ae None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:46.233459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f1842-4bc6-4fb1-9b8e-bf558ccf72ae None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:46.233459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f1842-4bc6-4fb1-9b8e-bf558ccf72ae None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:46.234790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:46.239884 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c182a-7034-4b5c-b1b4-4dd37d232083 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:46.240320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c182a-7034-4b5c-b1b4-4dd37d232083 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:46.240451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c182a-7034-4b5c-b1b4-4dd37d232083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:46.305796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-577c182a-7034-4b5c-b1b4-4dd37d232083 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:46.307023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-577c182a-7034-4b5c-b1b4-4dd37d232083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:46.319805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:46.400978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:46.689192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f1842-4bc6-4fb1-9b8e-bf558ccf72ae None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:46.691019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e28f1842-4bc6-4fb1-9b8e-bf558ccf72ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:46.792172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f1842-4bc6-4fb1-9b8e-bf558ccf72ae None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:46.804571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:46.863709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b4045-cc29-45ed-ad3b-958118247905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:46.864314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b4045-cc29-45ed-ad3b-958118247905 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:46.864590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b4045-cc29-45ed-ad3b-958118247905 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:46.864858 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b4045-cc29-45ed-ad3b-958118247905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:46.866472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e59b4045-cc29-45ed-ad3b-958118247905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:46.973039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b4045-cc29-45ed-ad3b-958118247905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:46.974517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b4045-cc29-45ed-ad3b-958118247905 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:46.992333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 731/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:49.542166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bc39a-4ccf-4627-b023-ff4e1b37d566 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:49.542765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bc39a-4ccf-4627-b023-ff4e1b37d566 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:49.542765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bc39a-4ccf-4627-b023-ff4e1b37d566 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:49.544675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:36:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:49.549917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a5e05-4376-4481-b07c-db06e60b88c6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:49.550369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a5e05-4376-4481-b07c-db06e60b88c6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:49.550490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a5e05-4376-4481-b07c-db06e60b88c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:49.599971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:49.617715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-672a5e05-4376-4481-b07c-db06e60b88c6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:49.618938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-672a5e05-4376-4481-b07c-db06e60b88c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:49.713314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:49.994670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bc39a-4ccf-4627-b023-ff4e1b37d566 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:49.996242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-328bc39a-4ccf-4627-b023-ff4e1b37d566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:50.098352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bc39a-4ccf-4627-b023-ff4e1b37d566 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:50.138297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:50.170886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b92c5-83eb-4351-bfa5-6c0505fb10a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:50.171445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b92c5-83eb-4351-bfa5-6c0505fb10a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:50.171724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b92c5-83eb-4351-bfa5-6c0505fb10a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:50.172247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b92c5-83eb-4351-bfa5-6c0505fb10a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:50.174410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-555b92c5-83eb-4351-bfa5-6c0505fb10a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:50.279422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b92c5-83eb-4351-bfa5-6c0505fb10a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:50.280825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b92c5-83eb-4351-bfa5-6c0505fb10a6 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:50.299826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 717/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:52.587626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84c2eb-327a-4deb-8118-7b59d6f2b4b8 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:52.588202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84c2eb-327a-4deb-8118-7b59d6f2b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:52.588202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84c2eb-327a-4deb-8118-7b59d6f2b4b8 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:52.589508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:52.595449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54562d6-34db-45c9-b22d-beb2932a31fd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:52.595874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54562d6-34db-45c9-b22d-beb2932a31fd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:52.595988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54562d6-34db-45c9-b22d-beb2932a31fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:52.660111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c54562d6-34db-45c9-b22d-beb2932a31fd None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:52.661313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c54562d6-34db-45c9-b22d-beb2932a31fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:52.681799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:52.756116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:53.042617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84c2eb-327a-4deb-8118-7b59d6f2b4b8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:53.044167 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1b84c2eb-327a-4deb-8118-7b59d6f2b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:53.146484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84c2eb-327a-4deb-8118-7b59d6f2b4b8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:53.166513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:53.219162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff168c-daf5-4f24-90f7-d6db860094ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:53.219768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff168c-daf5-4f24-90f7-d6db860094ab service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:53.220048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff168c-daf5-4f24-90f7-d6db860094ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:53.220324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff168c-daf5-4f24-90f7-d6db860094ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:53.221918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-eeff168c-daf5-4f24-90f7-d6db860094ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:53.328745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff168c-daf5-4f24-90f7-d6db860094ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:53.330104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff168c-daf5-4f24-90f7-d6db860094ab service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:53.347708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 734/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:55.556447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeaa15-5bc5-47f4-b234-54fc04169225 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:55.557019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeaa15-5bc5-47f4-b234-54fc04169225 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:55.557019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaeaa15-5bc5-47f4-b234-54fc04169225 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:55.558384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:55.563840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4bdc8-85dd-465b-a9b0-55dd35832b61 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:55.564272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4bdc8-85dd-465b-a9b0-55dd35832b61 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:55.564411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4bdc8-85dd-465b-a9b0-55dd35832b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:55.626848 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:55.629626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e0e4bdc8-85dd-465b-a9b0-55dd35832b61 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:55.630846 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e0e4bdc8-85dd-465b-a9b0-55dd35832b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:55.725830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:56.010904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeaa15-5bc5-47f4-b234-54fc04169225 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:56.012976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-caaeaa15-5bc5-47f4-b234-54fc04169225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:56.115111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaeaa15-5bc5-47f4-b234-54fc04169225 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:56.163310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:56.187100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4c8d5-825e-49c6-9712-198c830b622a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:56.187697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4c8d5-825e-49c6-9712-198c830b622a service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:56.187973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4c8d5-825e-49c6-9712-198c830b622a service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:56.188242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4c8d5-825e-49c6-9712-198c830b622a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:56.189829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e1e4c8d5-825e-49c6-9712-198c830b622a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:56.295487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4c8d5-825e-49c6-9712-198c830b622a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:56.296902 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4c8d5-825e-49c6-9712-198c830b622a service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:56.316527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 720/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:36:58.510968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9cf56-1c2e-4e80-b134-3390f40d8b19 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:58.511422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9cf56-1c2e-4e80-b134-3390f40d8b19 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:58.511563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9cf56-1c2e-4e80-b134-3390f40d8b19 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:58.512894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:36:58.518305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000de1f3-0f11-4160-aa47-a717f29f3dc1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:58.518754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000de1f3-0f11-4160-aa47-a717f29f3dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:58.518850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000de1f3-0f11-4160-aa47-a717f29f3dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:58.583071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-000de1f3-0f11-4160-aa47-a717f29f3dc1 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:36:58.584269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-000de1f3-0f11-4160-aa47-a717f29f3dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:58.613516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:58.678829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:36:58.961316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9cf56-1c2e-4e80-b134-3390f40d8b19 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:36:58.962895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9bd9cf56-1c2e-4e80-b134-3390f40d8b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:59.063745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9cf56-1c2e-4e80-b134-3390f40d8b19 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:36:59.092767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:36:59.136050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa173a64-503e-4236-b831-137830f3e33d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:36:59.136664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa173a64-503e-4236-b831-137830f3e33d service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:36:59.137544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa173a64-503e-4236-b831-137830f3e33d service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:36:59.137815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa173a64-503e-4236-b831-137830f3e33d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:36:59.139439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aa173a64-503e-4236-b831-137830f3e33d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:36:59.244640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa173a64-503e-4236-b831-137830f3e33d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:36:59.245979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa173a64-503e-4236-b831-137830f3e33d service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:36:59.264422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 737/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:37:01.462058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330750f-6bdb-4a0f-9bc5-5bddfc3bd412 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:01.462664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330750f-6bdb-4a0f-9bc5-5bddfc3bd412 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:01.462664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330750f-6bdb-4a0f-9bc5-5bddfc3bd412 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:01.464044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:01.469947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e3230-7d21-4011-a831-a4605fcd6283 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:01.470393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e3230-7d21-4011-a831-a4605fcd6283 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:01.470533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e3230-7d21-4011-a831-a4605fcd6283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:01.535633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-298e3230-7d21-4011-a831-a4605fcd6283 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:01.536834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-298e3230-7d21-4011-a831-a4605fcd6283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:01.545470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:01.630531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:37:01.915160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330750f-6bdb-4a0f-9bc5-5bddfc3bd412 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:01.916729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5330750f-6bdb-4a0f-9bc5-5bddfc3bd412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:02.020141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330750f-6bdb-4a0f-9bc5-5bddfc3bd412 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:02.074642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:02.093101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b6788-5162-4bf9-a247-574cca4d8cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:02.093712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b6788-5162-4bf9-a247-574cca4d8cef service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:02.093990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b6788-5162-4bf9-a247-574cca4d8cef service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:02.094344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b6788-5162-4bf9-a247-574cca4d8cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:02.095934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3a6b6788-5162-4bf9-a247-574cca4d8cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:02.201517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b6788-5162-4bf9-a247-574cca4d8cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:02.202883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b6788-5162-4bf9-a247-574cca4d8cef service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:02.222227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 723/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:37:04.392231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db3c0e-3dd8-48c4-a362-1c1f461737ee None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:04.392784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db3c0e-3dd8-48c4-a362-1c1f461737ee None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:04.392784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db3c0e-3dd8-48c4-a362-1c1f461737ee None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:04.394157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:04.399513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a762e-c1f5-49d6-a4bf-823efedc0127 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:04.399933 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a762e-c1f5-49d6-a4bf-823efedc0127 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:04.400086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a762e-c1f5-49d6-a4bf-823efedc0127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:04.465042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f25a762e-c1f5-49d6-a4bf-823efedc0127 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:04.466281 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f25a762e-c1f5-49d6-a4bf-823efedc0127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:04.504430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:04.561862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:37:04.848153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db3c0e-3dd8-48c4-a362-1c1f461737ee None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:04.849720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-71db3c0e-3dd8-48c4-a362-1c1f461737ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:04.957491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db3c0e-3dd8-48c4-a362-1c1f461737ee None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:04.994704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:05.030196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843d844-ec96-47a3-9196-55a48ada2fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:05.030808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843d844-ec96-47a3-9196-55a48ada2fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:05.031081 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843d844-ec96-47a3-9196-55a48ada2fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:05.031349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843d844-ec96-47a3-9196-55a48ada2fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:05.032965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d843d844-ec96-47a3-9196-55a48ada2fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:05.138071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843d844-ec96-47a3-9196-55a48ada2fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:05.139452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843d844-ec96-47a3-9196-55a48ada2fe5 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:05.157602 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 740/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:37:07.345079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a803bd-6d2b-4cdd-9f0d-b84d272aa35c None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:07.345635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a803bd-6d2b-4cdd-9f0d-b84d272aa35c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:07.345635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a803bd-6d2b-4cdd-9f0d-b84d272aa35c None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:07.347065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:07.352501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf76e4c-d761-411b-ba5a-8d987e8c317f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:07.352904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf76e4c-d761-411b-ba5a-8d987e8c317f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:07.353042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf76e4c-d761-411b-ba5a-8d987e8c317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:07.418957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7bf76e4c-d761-411b-ba5a-8d987e8c317f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:07.420157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7bf76e4c-d761-411b-ba5a-8d987e8c317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:07.438393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:07.517657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:37:07.801586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a803bd-6d2b-4cdd-9f0d-b84d272aa35c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:07.803185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7a803bd-6d2b-4cdd-9f0d-b84d272aa35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:07.908381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a803bd-6d2b-4cdd-9f0d-b84d272aa35c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:07.973862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:07.979959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91397d-deae-419f-8943-a60dce6121bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:07.981049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91397d-deae-419f-8943-a60dce6121bd service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:07.981321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91397d-deae-419f-8943-a60dce6121bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:07.981585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91397d-deae-419f-8943-a60dce6121bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:07.983261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0c91397d-deae-419f-8943-a60dce6121bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:08.089062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91397d-deae-419f-8943-a60dce6121bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:08.090948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91397d-deae-419f-8943-a60dce6121bd service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:08.108836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 726/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:37:10.258753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c61a7-b3fa-465e-972b-c265bd85eab2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:10.259282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c61a7-b3fa-465e-972b-c265bd85eab2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:10.259282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c61a7-b3fa-465e-972b-c265bd85eab2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:10.260556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:10.265807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f998b-8fd3-49af-a7be-b20adce1c516 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:10.266272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f998b-8fd3-49af-a7be-b20adce1c516 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:10.266379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f998b-8fd3-49af-a7be-b20adce1c516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:10.332398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cc1f998b-8fd3-49af-a7be-b20adce1c516 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:10.333606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc1f998b-8fd3-49af-a7be-b20adce1c516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:10.380422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:10.428481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:37:10.710258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c61a7-b3fa-465e-972b-c265bd85eab2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:10.712075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c68c61a7-b3fa-465e-972b-c265bd85eab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:10.813671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c61a7-b3fa-465e-972b-c265bd85eab2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:10.861609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:10.887904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa42c4-9633-4308-a6d3-a4c8af0f1931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:10.888514 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa42c4-9633-4308-a6d3-a4c8af0f1931 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:10.888790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa42c4-9633-4308-a6d3-a4c8af0f1931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:10.889051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa42c4-9633-4308-a6d3-a4c8af0f1931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:10.890691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b4aa42c4-9633-4308-a6d3-a4c8af0f1931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:10.996566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa42c4-9633-4308-a6d3-a4c8af0f1931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:10.998453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa42c4-9633-4308-a6d3-a4c8af0f1931 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:11.016959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 743/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:37:20.426943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=22 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:21.014755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=66 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:31.612401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6967b05-6e31-4b05-b64f-a453e29cb0f9 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:31.612992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6967b05-6e31-4b05-b64f-a453e29cb0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:31.612992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6967b05-6e31-4b05-b64f-a453e29cb0f9 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:31.614335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 728/1471] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:31.619970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4020a-112c-4fdb-b1a1-79b06d7ea6a4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:31.620407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4020a-112c-4fdb-b1a1-79b06d7ea6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:31.620517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4020a-112c-4fdb-b1a1-79b06d7ea6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:31.687305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-94e4020a-112c-4fdb-b1a1-79b06d7ea6a4 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:31.688504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-94e4020a-112c-4fdb-b1a1-79b06d7ea6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:31.783076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 744/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:37:32.080446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38c797-1a78-4553-b614-b6b35c678573 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:32.080731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38c797-1a78-4553-b614-b6b35c678573 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:32.081006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38c797-1a78-4553-b614-b6b35c678573 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:32.082366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 729/1473] 10.222.0.26 () {54 vars in 962 bytes} [Tue Oct 4 05:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:32.087303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a9c70-6758-4f65-a6ac-bbfa5ce15b3f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:32.087582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a9c70-6758-4f65-a6ac-bbfa5ce15b3f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:32.087878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a9c70-6758-4f65-a6ac-bbfa5ce15b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:32.162852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-841a9c70-6758-4f65-a6ac-bbfa5ce15b3f None None] MFA Rules not processed for user `c34eb8fcef0543c282b3887dfeb74c15`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:32.164098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-841a9c70-6758-4f65-a6ac-bbfa5ce15b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:32.184596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:32.239239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 745/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Oct 4 05:37:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:37:32.246890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38c797-1a78-4553-b614-b6b35c678573 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:32.248419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6d38c797-1a78-4553-b614-b6b35c678573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:32.350533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38c797-1a78-4553-b614-b6b35c678573 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:32.421449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e0986-6ffa-495f-8613-e55ebe135fd5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34eb8fcef0543c282b3887dfeb74c15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:32.421979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e0986-6ffa-495f-8613-e55ebe135fd5 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:32.422284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e0986-6ffa-495f-8613-e55ebe135fd5 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:32.422554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e0986-6ffa-495f-8613-e55ebe135fd5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:32.424128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-401e0986-6ffa-495f-8613-e55ebe135fd5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:32.477535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:32.532276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e0986-6ffa-495f-8613-e55ebe135fd5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:32.533703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e0986-6ffa-495f-8613-e55ebe135fd5 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:32.551590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 730/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:37:34.796949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91e636-0e7d-4898-a120-c942d29b56b2 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:34.797461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91e636-0e7d-4898-a120-c942d29b56b2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:34.797461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91e636-0e7d-4898-a120-c942d29b56b2 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:34.798773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 746/1476] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:34.804068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433eae-0cf0-46d4-beec-866d27417672 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:34.804485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433eae-0cf0-46d4-beec-866d27417672 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:34.804626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433eae-0cf0-46d4-beec-866d27417672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:34.871607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-19433eae-0cf0-46d4-beec-866d27417672 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:34.872803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-19433eae-0cf0-46d4-beec-866d27417672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:34.965802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 731/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:37:35.241367 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91e636-0e7d-4898-a120-c942d29b56b2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:35.242949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3b91e636-0e7d-4898-a120-c942d29b56b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:35.364283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91e636-0e7d-4898-a120-c942d29b56b2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:35.414847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:35.458840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7564ab-7e17-4d0d-8d14-aa2c3f7bcdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:35.459443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7564ab-7e17-4d0d-8d14-aa2c3f7bcdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:35.459714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7564ab-7e17-4d0d-8d14-aa2c3f7bcdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:35.459978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7564ab-7e17-4d0d-8d14-aa2c3f7bcdb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:35.460601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7564ab-7e17-4d0d-8d14-aa2c3f7bcdb4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:35.462339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-be7564ab-7e17-4d0d-8d14-aa2c3f7bcdb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:37:35.462339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Oct 04 05:37:35.462339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: Oct 04 05:37:35.463461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7564ab-7e17-4d0d-8d14-aa2c3f7bcdb4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:35.482347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 747/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 05:37:35 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:37:37.643963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500e6a0-371d-4ab1-ac65-a820404a6d9e None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:37.644554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500e6a0-371d-4ab1-ac65-a820404a6d9e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:37.644554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500e6a0-371d-4ab1-ac65-a820404a6d9e None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:37.645841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 732/1479] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:37.651709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df058899-7306-4c78-a962-943a465de253 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:37.652144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df058899-7306-4c78-a962-943a465de253 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:37.652248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df058899-7306-4c78-a962-943a465de253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:37.716969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-df058899-7306-4c78-a962-943a465de253 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:37.718199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-df058899-7306-4c78-a962-943a465de253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:37.810562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 748/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:37:38.102034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500e6a0-371d-4ab1-ac65-a820404a6d9e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:38.103597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e500e6a0-371d-4ab1-ac65-a820404a6d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:38.113274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:38.246589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500e6a0-371d-4ab1-ac65-a820404a6d9e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:38.324015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:38.354951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ae6a7-a491-44a7-8c0c-7bdfba41b032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:38.355558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ae6a7-a491-44a7-8c0c-7bdfba41b032 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:38.356008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ae6a7-a491-44a7-8c0c-7bdfba41b032 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:38.356568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ae6a7-a491-44a7-8c0c-7bdfba41b032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:38.358174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-569ae6a7-a491-44a7-8c0c-7bdfba41b032 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:38.445381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ae6a7-a491-44a7-8c0c-7bdfba41b032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:38.447013 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ae6a7-a491-44a7-8c0c-7bdfba41b032 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:38.464903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 733/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:37:40.782474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9e660-cd82-4ed6-8c7b-fd0134f41453 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:40.783024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9e660-cd82-4ed6-8c7b-fd0134f41453 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:40.783024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9e660-cd82-4ed6-8c7b-fd0134f41453 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:40.784316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 749/1482] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:40.789795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bf146-1785-4c6a-845f-94bc77519749 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:40.790251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bf146-1785-4c6a-845f-94bc77519749 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:40.790382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bf146-1785-4c6a-845f-94bc77519749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:40.857075 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1a9bf146-1785-4c6a-845f-94bc77519749 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:40.858326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1a9bf146-1785-4c6a-845f-94bc77519749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:40.905159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:40.955059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 734/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:37:41.238524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9e660-cd82-4ed6-8c7b-fd0134f41453 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:41.240061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-56d9e660-cd82-4ed6-8c7b-fd0134f41453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:41.263029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:41.341424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9e660-cd82-4ed6-8c7b-fd0134f41453 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:41.412664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454cf9c-5bdb-45c0-b1d4-86f175863795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:41.413268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454cf9c-5bdb-45c0-b1d4-86f175863795 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:41.413547 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454cf9c-5bdb-45c0-b1d4-86f175863795 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:41.413821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454cf9c-5bdb-45c0-b1d4-86f175863795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:41.415435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1454cf9c-5bdb-45c0-b1d4-86f175863795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:41.489476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:41.523663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454cf9c-5bdb-45c0-b1d4-86f175863795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:41.525113 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454cf9c-5bdb-45c0-b1d4-86f175863795 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:41.543209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 750/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:37:43.734112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84fe98-10fb-42b5-a3b1-f05c5070147a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:43.734688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84fe98-10fb-42b5-a3b1-f05c5070147a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:43.734688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84fe98-10fb-42b5-a3b1-f05c5070147a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:43.735992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 735/1485] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:43.741126 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb6ddd-617d-4891-8f9e-789b46b83773 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:43.741588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb6ddd-617d-4891-8f9e-789b46b83773 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:43.741708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb6ddd-617d-4891-8f9e-789b46b83773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:43.805985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8bbb6ddd-617d-4891-8f9e-789b46b83773 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:43.807824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8bbb6ddd-617d-4891-8f9e-789b46b83773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:43.901974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 751/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:37:44.189305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84fe98-10fb-42b5-a3b1-f05c5070147a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:44.191145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3c84fe98-10fb-42b5-a3b1-f05c5070147a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:44.292923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84fe98-10fb-42b5-a3b1-f05c5070147a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:44.362815 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:44.364276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de358fd0-4f9a-4fd3-a707-c18aba7d7ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:44.364872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de358fd0-4f9a-4fd3-a707-c18aba7d7ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:44.365150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de358fd0-4f9a-4fd3-a707-c18aba7d7ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:44.365419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de358fd0-4f9a-4fd3-a707-c18aba7d7ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:44.367039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-de358fd0-4f9a-4fd3-a707-c18aba7d7ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:44.473680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de358fd0-4f9a-4fd3-a707-c18aba7d7ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:44.475061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de358fd0-4f9a-4fd3-a707-c18aba7d7ba8 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:44.493190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 736/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:37:46.788733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777045d8-d4d1-43bd-8127-2235667d03be None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:46.789261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777045d8-d4d1-43bd-8127-2235667d03be None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:46.789261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777045d8-d4d1-43bd-8127-2235667d03be None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:46.791121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 752/1488] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:46.796272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f04482-6df5-4b48-bbab-277d213cb614 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:46.796700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f04482-6df5-4b48-bbab-277d213cb614 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:46.796819 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f04482-6df5-4b48-bbab-277d213cb614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:46.863374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b3f04482-6df5-4b48-bbab-277d213cb614 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:46.864595 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b3f04482-6df5-4b48-bbab-277d213cb614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:46.917254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:46.960185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 737/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:37:47.250632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777045d8-d4d1-43bd-8127-2235667d03be None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:47.252187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-777045d8-d4d1-43bd-8127-2235667d03be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:47.287565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:47.354339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777045d8-d4d1-43bd-8127-2235667d03be None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:47.425157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d68370-d947-4484-b309-4a16fa47648c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:47.425750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d68370-d947-4484-b309-4a16fa47648c service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:47.426025 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d68370-d947-4484-b309-4a16fa47648c service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:47.426325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d68370-d947-4484-b309-4a16fa47648c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:47.427898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-20d68370-d947-4484-b309-4a16fa47648c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:47.506973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:47.537046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d68370-d947-4484-b309-4a16fa47648c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:47.538555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d68370-d947-4484-b309-4a16fa47648c service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:47.556662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 753/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:37:50.040214 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fcac5-1878-4b9c-9a52-c4f9d5a107ef None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:50.041263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fcac5-1878-4b9c-9a52-c4f9d5a107ef None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:50.041263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fcac5-1878-4b9c-9a52-c4f9d5a107ef None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:50.042667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 738/1491] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 05:37:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:37:50.047838 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f515f-d89f-4243-b550-4f5563905bd6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:50.048256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f515f-d89f-4243-b550-4f5563905bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:50.048395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f515f-d89f-4243-b550-4f5563905bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:50.113817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-686f515f-d89f-4243-b550-4f5563905bd6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:37:50.115047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-686f515f-d89f-4243-b550-4f5563905bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:50.208797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 754/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 05:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:37:50.493510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fcac5-1878-4b9c-9a52-c4f9d5a107ef None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:37:50.495140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9f3fcac5-1878-4b9c-9a52-c4f9d5a107ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:50.598254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fcac5-1878-4b9c-9a52-c4f9d5a107ef None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:37:50.671397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de781bc-663e-457d-a985-d8f30347bf2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:37:50.671999 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de781bc-663e-457d-a985-d8f30347bf2f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:37:50.672273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de781bc-663e-457d-a985-d8f30347bf2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:37:50.672544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de781bc-663e-457d-a985-d8f30347bf2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:37:50.674125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2de781bc-663e-457d-a985-d8f30347bf2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:37:50.682563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:37:50.781548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de781bc-663e-457d-a985-d8f30347bf2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:37:50.782945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de781bc-663e-457d-a985-d8f30347bf2f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:37:50.800597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 739/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:38:00.207317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=82 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:38:00.798470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=50 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:11.410670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2214-f818-4ce6-a281-af5895531965 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:11.411259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2214-f818-4ce6-a281-af5895531965 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:11.411259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2214-f818-4ce6-a281-af5895531965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:11.485807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cc8b2214-f818-4ce6-a281-af5895531965 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:39:11.487044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cc8b2214-f818-4ce6-a281-af5895531965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:11.580991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 755/1494] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:39:11.589481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9d90-cdd9-47e8-9c4b-b329cd46e6da None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:11.589988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9d90-cdd9-47e8-9c4b-b329cd46e6da None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:11.590104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9d90-cdd9-47e8-9c4b-b329cd46e6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:11.665478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e8fd9d90-cdd9-47e8-9c4b-b329cd46e6da None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:39:11.666657 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e8fd9d90-cdd9-47e8-9c4b-b329cd46e6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:11.760593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 740/1495] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:39:11.765220 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b2214-f818-4ce6-a281-af5895531965 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:39:11.766536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cc8b2214-f818-4ce6-a281-af5895531965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:11.778388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b2214-f818-4ce6-a281-af5895531965 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:39:11.839813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:11.871827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:39:11.872437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:11.872715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:11.872983 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:11.873617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:39:11.873922 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:39:11.874960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:39:11.874960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Oct 04 05:39:11.874960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: Oct 04 05:39:11.875845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:39:11.891492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 756/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:39:11.896988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9d90-cdd9-47e8-9c4b-b329cd46e6da None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:39:11.906268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9d90-cdd9-47e8-9c4b-b329cd46e6da None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:39:11.967431 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:12.000181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e65077-1038-4c75-bc4b-17f955dd5885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:39:12.000781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e65077-1038-4c75-bc4b-17f955dd5885 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:12.001056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e65077-1038-4c75-bc4b-17f955dd5885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:12.001323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e65077-1038-4c75-bc4b-17f955dd5885 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:12.001952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e65077-1038-4c75-bc4b-17f955dd5885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:39:12.003714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-79e65077-1038-4c75-bc4b-17f955dd5885 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:39:12.003714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 04 05:39:12.003714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: Oct 04 05:39:12.004900 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e65077-1038-4c75-bc4b-17f955dd5885 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:39:12.104430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 741/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:39:11 2022] POST /identity/v3/projects => generated 377 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:39:12.108560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:39:12.110069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:12.142562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9ec5e-b48a-4de4-a8ea-cc94f230ca9a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:39:12.234702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc649e-876d-453f-a2e7-34a839caa194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:39:12.235222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc649e-876d-453f-a2e7-34a839caa194 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:12.235493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc649e-876d-453f-a2e7-34a839caa194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:12.235759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc649e-876d-453f-a2e7-34a839caa194 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:12.236385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc649e-876d-453f-a2e7-34a839caa194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:39:12.237383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-a8bc649e-876d-453f-a2e7-34a839caa194 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:39:12.237383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Oct 04 05:39:12.237383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: Oct 04 05:39:12.238618 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc649e-876d-453f-a2e7-34a839caa194 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:39:12.345886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 757/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:39:12 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:39:12.350512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e65077-1038-4c75-bc4b-17f955dd5885 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:39:12.352101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-79e65077-1038-4c75-bc4b-17f955dd5885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:12.412040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e65077-1038-4c75-bc4b-17f955dd5885 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:39:12.495093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:12.504794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f41dd2-9c77-4d2f-b9f8-649eca9de4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:39:12.506944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f41dd2-9c77-4d2f-b9f8-649eca9de4ba admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:12.506944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f41dd2-9c77-4d2f-b9f8-649eca9de4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:12.506944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f41dd2-9c77-4d2f-b9f8-649eca9de4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:12.506944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f41dd2-9c77-4d2f-b9f8-649eca9de4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:39:12.507581 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-a1f41dd2-9c77-4d2f-b9f8-649eca9de4ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:39:12.507581 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 04 05:39:12.507581 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: Oct 04 05:39:12.508475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f41dd2-9c77-4d2f-b9f8-649eca9de4ba admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:39:12.522445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 742/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:39:12.526528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc649e-876d-453f-a2e7-34a839caa194 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:39:12.528051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a8bc649e-876d-453f-a2e7-34a839caa194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:12.539899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc649e-876d-453f-a2e7-34a839caa194 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:39:12.565010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:12.638155 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315979b0-fff1-44e0-9eaf-211db5381143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:39:12.638771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315979b0-fff1-44e0-9eaf-211db5381143 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:12.639044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315979b0-fff1-44e0-9eaf-211db5381143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:12.639302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315979b0-fff1-44e0-9eaf-211db5381143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:12.639940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315979b0-fff1-44e0-9eaf-211db5381143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:39:12.640938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-315979b0-fff1-44e0-9eaf-211db5381143 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:39:12.640938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Oct 04 05:39:12.640938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: Oct 04 05:39:12.641844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315979b0-fff1-44e0-9eaf-211db5381143 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:39:12.655102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 758/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:39:12.659631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f41dd2-9c77-4d2f-b9f8-649eca9de4ba admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:39:12.661182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a1f41dd2-9c77-4d2f-b9f8-649eca9de4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:12.706673 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f41dd2-9c77-4d2f-b9f8-649eca9de4ba admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:39:12.801064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286314d-f43e-44b6-ba46-292270b9fe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:39:12.801727 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286314d-f43e-44b6-ba46-292270b9fe43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:12.802014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286314d-f43e-44b6-ba46-292270b9fe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:12.802313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286314d-f43e-44b6-ba46-292270b9fe43 admin admin] PATH_INFO: `/v3/projects/a389f4d979bb4e60a523ce81f11f4d3b/users/56b274657b914e6cbf6749f10bf6b7c7/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:12.824791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286314d-f43e-44b6-ba46-292270b9fe43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a389f4d979bb4e60a523ce81f11f4d3b, user_id=56b274657b914e6cbf6749f10bf6b7c7, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:39:12.825814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-9286314d-f43e-44b6-ba46-292270b9fe43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:39:12.825814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 04 05:39:12.825814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: Oct 04 05:39:12.827782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286314d-f43e-44b6-ba46-292270b9fe43 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:39:12.926230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 743/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:39:12 2022] PUT /identity/v3/projects/a389f4d979bb4e60a523ce81f11f4d3b/users/56b274657b914e6cbf6749f10bf6b7c7/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:39:12.947923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc558962-169e-471c-b59a-22ee8434cdee None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:12.948230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc558962-169e-471c-b59a-22ee8434cdee None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:12.948541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc558962-169e-471c-b59a-22ee8434cdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:13.005104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bc558962-169e-471c-b59a-22ee8434cdee None None] MFA Rules not processed for user `56b274657b914e6cbf6749f10bf6b7c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:39:13.006835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bc558962-169e-471c-b59a-22ee8434cdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:13.085954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 759/1502] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:39:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:39:13.099343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93f1a1-8fcf-4bd5-ab63-1a3718fae370 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:13.099631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93f1a1-8fcf-4bd5-ab63-1a3718fae370 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:13.099927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93f1a1-8fcf-4bd5-ab63-1a3718fae370 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:13.101249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 744/1503] 10.222.0.26 () {54 vars in 963 bytes} [Tue Oct 4 05:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:39:13.105988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c16cb7-74f2-439b-ac1a-df753f9100e2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:13.106311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c16cb7-74f2-439b-ac1a-df753f9100e2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:13.106586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c16cb7-74f2-439b-ac1a-df753f9100e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:13.137319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:13.181692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-53c16cb7-74f2-439b-ac1a-df753f9100e2 None None] MFA Rules not processed for user `131afaf047d34e88842fffd0d0b76242`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:39:13.182942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-53c16cb7-74f2-439b-ac1a-df753f9100e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:13.257134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 760/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Tue Oct 4 05:39:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:39:13.264881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93f1a1-8fcf-4bd5-ab63-1a3718fae370 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:39:13.266453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0b93f1a1-8fcf-4bd5-ab63-1a3718fae370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:13.316505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:13.369740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93f1a1-8fcf-4bd5-ab63-1a3718fae370 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:39:13.444909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a0170-71d5-448f-adf8-a80d1f600924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:39:13.445521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a0170-71d5-448f-adf8-a80d1f600924 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:13.445797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a0170-71d5-448f-adf8-a80d1f600924 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:13.446067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a0170-71d5-448f-adf8-a80d1f600924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:13.447759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cf1a0170-71d5-448f-adf8-a80d1f600924 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:13.544799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a0170-71d5-448f-adf8-a80d1f600924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:39:13.546252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a0170-71d5-448f-adf8-a80d1f600924 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:39:13.549677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:13.564712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 745/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:39:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:39:13.609727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2c49-59cd-4d52-a762-ec333609ecba None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:13.610129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2c49-59cd-4d52-a762-ec333609ecba None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:13.610319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2c49-59cd-4d52-a762-ec333609ecba None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:13.611609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 761/1506] 10.222.0.26 () {52 vars in 830 bytes} [Tue Oct 4 05:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:39:13.615141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a0170-71d5-448f-adf8-a80d1f600924 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:39:13.616880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cf1a0170-71d5-448f-adf8-a80d1f600924 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:13.714872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a0170-71d5-448f-adf8-a80d1f600924 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:39:13.806358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:39:13.806984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:13.807259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:13.807531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] PATH_INFO: `/v3/users/56b274657b914e6cbf6749f10bf6b7c7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:13.820469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b274657b914e6cbf6749f10bf6b7c7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:39:13.821474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:39:13.821474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 04 05:39:13.821474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: Oct 04 05:39:13.822705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:39:13.832180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:14.028398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b274657b914e6cbf6749f10bf6b7c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:39:14.028760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b274657b914e6cbf6749f10bf6b7c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:39:14.052089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b274657b914e6cbf6749f10bf6b7c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:39:14.111433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b274657b914e6cbf6749f10bf6b7c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:39:14.111947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b274657b914e6cbf6749f10bf6b7c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:39:14.131054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 746/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:39:13 2022] DELETE /identity/v3/users/56b274657b914e6cbf6749f10bf6b7c7 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:39:14.135537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d431d-0634-4873-a672-23d34859a188 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:14.135820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d431d-0634-4873-a672-23d34859a188 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:14.136092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d431d-0634-4873-a672-23d34859a188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:14.193075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-209d431d-0634-4873-a672-23d34859a188 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:39:14.194320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-209d431d-0634-4873-a672-23d34859a188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:14.267523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:14.280079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 762/1508] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:39:14.290240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:39:14.291474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:14.390503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f1200-dc20-4881-9364-0eac51f652de admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:39:14.446125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:14.465339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190df921-ed90-44ef-89a7-a42080999304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:39:14.468089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190df921-ed90-44ef-89a7-a42080999304 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:14.468089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190df921-ed90-44ef-89a7-a42080999304 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:14.468089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190df921-ed90-44ef-89a7-a42080999304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:14.468089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-190df921-ed90-44ef-89a7-a42080999304 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:14.576743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190df921-ed90-44ef-89a7-a42080999304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:39:14.578088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190df921-ed90-44ef-89a7-a42080999304 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:39:14.596597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 747/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:39:15.132123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d431d-0634-4873-a672-23d34859a188 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:39:15.132123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-209d431d-0634-4873-a672-23d34859a188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:39:15.132123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d431d-0634-4873-a672-23d34859a188 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:39:15.163959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:39:15.164622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:39:15.164898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:39:15.165168 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] PATH_INFO: `/v3/projects/a389f4d979bb4e60a523ce81f11f4d3b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:39:15.174904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a389f4d979bb4e60a523ce81f11f4d3b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:39:15.175917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:39:15.175917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Oct 04 05:39:15.175917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: Oct 04 05:39:15.177110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:39:15.292640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a389f4d979bb4e60a523ce81f11f4d3b', 'request_id': 'req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:39:15.306088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a389f4d979bb4e60a523ce81f11f4d3b', 'request_id': 'req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:39:15.373993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a389f4d979bb4e60a523ce81f11f4d3b', 'request_id': 'req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:39:15.396598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] The token cache is being invalidate because project a389f4d979bb4e60a523ce81f11f4d3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:39:15.397006 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:39:15.397550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-edce8f13-aa3f-4ef7-99d6-aebaa7a993e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:39:15.398881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 763/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:39:15 2022] DELETE /identity/v3/projects/a389f4d979bb4e60a523ce81f11f4d3b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:39:24.594393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=62 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:39:25.394490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:40:41.295658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6e81a-34bc-4ed6-8313-fa01c9203f7f None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:41.296218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6e81a-34bc-4ed6-8313-fa01c9203f7f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:41.296218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6e81a-34bc-4ed6-8313-fa01c9203f7f None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:41.297483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:40:41.302769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a935bc-451d-487d-9273-2f753d32db63 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:41.303176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a935bc-451d-487d-9273-2f753d32db63 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:41.303325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a935bc-451d-487d-9273-2f753d32db63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:41.383851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-23a935bc-451d-487d-9273-2f753d32db63 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:40:41.385216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-23a935bc-451d-487d-9273-2f753d32db63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:40:41.487404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:40:41.641915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6e81a-34bc-4ed6-8313-fa01c9203f7f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:40:41.644071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b3b6e81a-34bc-4ed6-8313-fa01c9203f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:40:41.762834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6e81a-34bc-4ed6-8313-fa01c9203f7f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:40:41.768349 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35caf17b-d8af-404a-b812-4387334015d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:40:41.768993 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caf17b-d8af-404a-b812-4387334015d8 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:41.769274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caf17b-d8af-404a-b812-4387334015d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:41.770086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caf17b-d8af-404a-b812-4387334015d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:41.778157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35caf17b-d8af-404a-b812-4387334015d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:40:41.779756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35caf17b-d8af-404a-b812-4387334015d8 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:40:41.789560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 749/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:40:42.858160 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d75267-897e-43cd-a339-6607921d9cb0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:42.858742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d75267-897e-43cd-a339-6607921d9cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:42.858742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d75267-897e-43cd-a339-6607921d9cb0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:42.860072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:40:42.866877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092a600-ce46-44ad-b79a-4a7880338729 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:42.867319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092a600-ce46-44ad-b79a-4a7880338729 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:42.867416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092a600-ce46-44ad-b79a-4a7880338729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:42.886536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1092a600-ce46-44ad-b79a-4a7880338729 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:40:42.887667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1092a600-ce46-44ad-b79a-4a7880338729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:40:42.895609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:40:43.048288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d75267-897e-43cd-a339-6607921d9cb0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:40:43.062838 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d75267-897e-43cd-a339-6607921d9cb0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:40:43.066608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc16a-8fb2-49a7-9b9b-77d99cb06504 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:43.066896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc16a-8fb2-49a7-9b9b-77d99cb06504 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:43.067177 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc16a-8fb2-49a7-9b9b-77d99cb06504 None None] PATH_INFO: `/v3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:43.068456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 751/1516] 10.222.0.39 () {54 vars in 925 bytes} [Tue Oct 4 05:40:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:40:43.068678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00f5d0-d168-4674-b7a3-f969c6a52c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:40:43.069266 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00f5d0-d168-4674-b7a3-f969c6a52c66 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:43.069544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00f5d0-d168-4674-b7a3-f969c6a52c66 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:43.069812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00f5d0-d168-4674-b7a3-f969c6a52c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:43.077721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00f5d0-d168-4674-b7a3-f969c6a52c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:40:43.079311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00f5d0-d168-4674-b7a3-f969c6a52c66 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:40:43.088442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 766/1517] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:40:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:40:43.098105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb17f9-3cc3-4fce-9131-4c4b201f5141 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:43.098516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb17f9-3cc3-4fce-9131-4c4b201f5141 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:43.098699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb17f9-3cc3-4fce-9131-4c4b201f5141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:43.164491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-41eb17f9-3cc3-4fce-9131-4c4b201f5141 None None] MFA Rules not processed for user `e95901cce5d448acb2571976163868a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:40:43.165660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-41eb17f9-3cc3-4fce-9131-4c4b201f5141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:40:43.225998 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:40:43.244099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 752/1518] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Oct 4 05:40:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:40:52.154857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590b23-1cdd-4772-bd42-3a6b073a8297 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:52.155328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590b23-1cdd-4772-bd42-3a6b073a8297 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:52.155461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590b23-1cdd-4772-bd42-3a6b073a8297 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:52.156823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 767/1519] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 05:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:40:52.162452 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60df01-6b8c-44c3-b1bb-ce9251b55ef4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:52.165560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60df01-6b8c-44c3-b1bb-ce9251b55ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:52.165560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60df01-6b8c-44c3-b1bb-ce9251b55ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:52.182809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff60df01-6b8c-44c3-b1bb-ce9251b55ef4 None None] MFA Rules not processed for user `e95901cce5d448acb2571976163868a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:40:52.184008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff60df01-6b8c-44c3-b1bb-ce9251b55ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:40:52.192831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 753/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 05:40:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:40:52.200285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7590b23-1cdd-4772-bd42-3a6b073a8297 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:40:52.213997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7590b23-1cdd-4772-bd42-3a6b073a8297 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:40:52.219096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f1017-682e-4648-8112-e07074e85ee0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:40:52.219689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1017-682e-4648-8112-e07074e85ee0 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:52.219969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1017-682e-4648-8112-e07074e85ee0 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:52.220242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1017-682e-4648-8112-e07074e85ee0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:52.229042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f1017-682e-4648-8112-e07074e85ee0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:40:52.230656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f1017-682e-4648-8112-e07074e85ee0 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:40:52.240157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 768/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:40:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:40:54.563318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12b80-fdca-451a-94b4-5ba99eb11935 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:54.564146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12b80-fdca-451a-94b4-5ba99eb11935 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:54.564146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb12b80-fdca-451a-94b4-5ba99eb11935 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:54.565454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:40:54.572181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e7791-3ea9-45f1-8e1e-df508c3a0a84 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:54.572586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e7791-3ea9-45f1-8e1e-df508c3a0a84 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:54.572728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e7791-3ea9-45f1-8e1e-df508c3a0a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:54.592125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b75e7791-3ea9-45f1-8e1e-df508c3a0a84 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:40:54.593316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b75e7791-3ea9-45f1-8e1e-df508c3a0a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:40:54.601153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:40:54.754256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb12b80-fdca-451a-94b4-5ba99eb11935 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:40:54.768884 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb12b80-fdca-451a-94b4-5ba99eb11935 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:40:54.774316 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b5e1b-624d-4044-8fed-55964a3c14fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:40:54.774910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5e1b-624d-4044-8fed-55964a3c14fa service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:54.775192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5e1b-624d-4044-8fed-55964a3c14fa service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:54.775461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5e1b-624d-4044-8fed-55964a3c14fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:54.783336 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b5e1b-624d-4044-8fed-55964a3c14fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:40:54.784893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b5e1b-624d-4044-8fed-55964a3c14fa service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:40:54.794646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 755/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:40:55.859602 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e830b6-4215-49de-bc59-58c7f3d05290 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:55.860181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e830b6-4215-49de-bc59-58c7f3d05290 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:55.860181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e830b6-4215-49de-bc59-58c7f3d05290 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:55.861512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 05:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:40:55.868228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750466f-fdb9-4551-bda2-fbec1bf1f7f1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:55.868689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750466f-fdb9-4551-bda2-fbec1bf1f7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:55.868795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750466f-fdb9-4551-bda2-fbec1bf1f7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:55.889280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2750466f-fdb9-4551-bda2-fbec1bf1f7f1 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:40:55.890519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2750466f-fdb9-4551-bda2-fbec1bf1f7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:40:55.898587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 05:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:40:56.049454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e830b6-4215-49de-bc59-58c7f3d05290 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:40:56.064398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e830b6-4215-49de-bc59-58c7f3d05290 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:40:56.070474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2c27f-29ed-448c-b637-6206d233bb70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:40:56.071097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2c27f-29ed-448c-b637-6206d233bb70 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:56.071376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2c27f-29ed-448c-b637-6206d233bb70 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:56.071648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2c27f-29ed-448c-b637-6206d233bb70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:56.079551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2c27f-29ed-448c-b637-6206d233bb70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:40:56.081117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2c27f-29ed-448c-b637-6206d233bb70 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:40:56.090270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 771/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:40:59.961314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c140c-9a21-446c-ab64-e9ad63d6c810 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:59.961857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c140c-9a21-446c-ab64-e9ad63d6c810 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:59.961857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c140c-9a21-446c-ab64-e9ad63d6c810 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:59.963292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:40:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:40:59.968756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afee314-a91c-42cb-afb8-3d928fd8b214 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:40:59.969219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afee314-a91c-42cb-afb8-3d928fd8b214 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:40:59.969330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afee314-a91c-42cb-afb8-3d928fd8b214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:40:59.989103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8afee314-a91c-42cb-afb8-3d928fd8b214 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:40:59.990338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8afee314-a91c-42cb-afb8-3d928fd8b214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:40:59.998208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:41:00.288937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c140c-9a21-446c-ab64-e9ad63d6c810 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:41:00.290686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e27c140c-9a21-446c-ab64-e9ad63d6c810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:41:00.368284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c140c-9a21-446c-ab64-e9ad63d6c810 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:41:00.372008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37792e0-6ed1-4368-8712-440316e57dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:41:00.372615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37792e0-6ed1-4368-8712-440316e57dee service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:41:00.372895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37792e0-6ed1-4368-8712-440316e57dee service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:41:00.373168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37792e0-6ed1-4368-8712-440316e57dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:41:00.382640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37792e0-6ed1-4368-8712-440316e57dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:41:00.384049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37792e0-6ed1-4368-8712-440316e57dee service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:41:00.393218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 758/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:41:02.526225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2a91e-7669-4042-bab4-0be36e0ebd2d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:41:02.526763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2a91e-7669-4042-bab4-0be36e0ebd2d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:41:02.526763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2a91e-7669-4042-bab4-0be36e0ebd2d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:41:02.528156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:41:02.533572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb67586-8b74-47d6-87b3-84f262bfea5d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:41:02.533982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb67586-8b74-47d6-87b3-84f262bfea5d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:41:02.534117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb67586-8b74-47d6-87b3-84f262bfea5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:41:02.554448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9fb67586-8b74-47d6-87b3-84f262bfea5d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:41:02.555635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9fb67586-8b74-47d6-87b3-84f262bfea5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:41:02.563330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:41:02.846583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2a91e-7669-4042-bab4-0be36e0ebd2d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:41:02.858477 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2a91e-7669-4042-bab4-0be36e0ebd2d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:41:02.862002 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5158e-5c34-455a-af69-b1ce10be0d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:41:02.862627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5158e-5c34-455a-af69-b1ce10be0d5b service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:41:02.862904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5158e-5c34-455a-af69-b1ce10be0d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:41:02.863172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5158e-5c34-455a-af69-b1ce10be0d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:41:02.871989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5158e-5c34-455a-af69-b1ce10be0d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:41:02.873447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5158e-5c34-455a-af69-b1ce10be0d5b service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:41:02.882696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 774/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:41:06.472980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cac7e-1994-410f-b7a1-d70095dbd618 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:41:06.473546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cac7e-1994-410f-b7a1-d70095dbd618 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:41:06.473546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cac7e-1994-410f-b7a1-d70095dbd618 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:41:06.474915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:41:06.480114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cdb64-5b6b-4fca-ad5b-86b7392cfaee None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:41:06.480541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cdb64-5b6b-4fca-ad5b-86b7392cfaee None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:41:06.481110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cdb64-5b6b-4fca-ad5b-86b7392cfaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:41:06.488456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:41:06.500731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-801cdb64-5b6b-4fca-ad5b-86b7392cfaee None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:41:06.504604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-801cdb64-5b6b-4fca-ad5b-86b7392cfaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:41:06.511774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:41:06.818031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cac7e-1994-410f-b7a1-d70095dbd618 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:41:06.819749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5c0cac7e-1994-410f-b7a1-d70095dbd618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:41:06.898509 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cac7e-1994-410f-b7a1-d70095dbd618 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:41:06.902225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f10c53-b2ae-4b5c-b144-725d170de92b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:41:06.902829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10c53-b2ae-4b5c-b144-725d170de92b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:41:06.903103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10c53-b2ae-4b5c-b144-725d170de92b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:41:06.903372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10c53-b2ae-4b5c-b144-725d170de92b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:41:06.912219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f10c53-b2ae-4b5c-b144-725d170de92b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:41:06.914214 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f10c53-b2ae-4b5c-b144-725d170de92b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:41:06.923393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 761/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:41:09.082327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2399a-5256-4668-b799-fa70e9901ef7 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:41:09.085895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2399a-5256-4668-b799-fa70e9901ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:41:09.085895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2399a-5256-4668-b799-fa70e9901ef7 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:41:09.085895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 05:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:41:09.090046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c0364-7359-4111-b34c-9104eba72aeb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:41:09.090473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c0364-7359-4111-b34c-9104eba72aeb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:41:09.090639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c0364-7359-4111-b34c-9104eba72aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:41:09.110213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b59c0364-7359-4111-b34c-9104eba72aeb None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:41:09.111422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b59c0364-7359-4111-b34c-9104eba72aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:41:09.119137 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 05:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:41:09.421859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2399a-5256-4668-b799-fa70e9901ef7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:41:09.423595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-96d2399a-5256-4668-b799-fa70e9901ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:41:09.436987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2399a-5256-4668-b799-fa70e9901ef7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:41:09.440815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc7af9-b205-41e0-9239-67cf4ce0539e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:41:09.441408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7af9-b205-41e0-9239-67cf4ce0539e service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:41:09.441685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7af9-b205-41e0-9239-67cf4ce0539e service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:41:09.441962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7af9-b205-41e0-9239-67cf4ce0539e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:41:09.451377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc7af9-b205-41e0-9239-67cf4ce0539e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:41:09.452808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc7af9-b205-41e0-9239-67cf4ce0539e service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:41:09.462183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 777/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:41:19.105476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:41:19.458056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=8 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:32.030843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0e9cd-9e35-4b1b-b207-b254f1c91a7a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.032498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0e9cd-9e35-4b1b-b207-b254f1c91a7a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.032498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0e9cd-9e35-4b1b-b207-b254f1c91a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.060402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f3c0e9cd-9e35-4b1b-b207-b254f1c91a7a None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:32.062024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f3c0e9cd-9e35-4b1b-b207-b254f1c91a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:32.074607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 763/1540] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:32.083832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad77064-2d2c-40af-95bc-0f256be8201b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.084391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad77064-2d2c-40af-95bc-0f256be8201b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.084487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad77064-2d2c-40af-95bc-0f256be8201b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.120383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8ad77064-2d2c-40af-95bc-0f256be8201b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:32.121632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8ad77064-2d2c-40af-95bc-0f256be8201b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:32.131223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 778/1541] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:42:32.136071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0e9cd-9e35-4b1b-b207-b254f1c91a7a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:32.151687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0e9cd-9e35-4b1b-b207-b254f1c91a7a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:32.159676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b0919-1baa-4ad2-8a54-df69f14c0f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:32.160535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b0919-1baa-4ad2-8a54-df69f14c0f84 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.161595 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b0919-1baa-4ad2-8a54-df69f14c0f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.162069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b0919-1baa-4ad2-8a54-df69f14c0f84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.162974 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b0919-1baa-4ad2-8a54-df69f14c0f84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:32.163495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b0919-1baa-4ad2-8a54-df69f14c0f84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:32.164735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-0d4b0919-1baa-4ad2-8a54-df69f14c0f84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:42:32.164735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 04 05:42:32.164735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: Oct 04 05:42:32.166091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b0919-1baa-4ad2-8a54-df69f14c0f84 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:32.173538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47412-8805-4008-bce5-eb9b28f68b80 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.173852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47412-8805-4008-bce5-eb9b28f68b80 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.174093 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c47412-8805-4008-bce5-eb9b28f68b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.187273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 764/1542] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:32.207461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-43c47412-8805-4008-bce5-eb9b28f68b80 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:32.208537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b0919-1baa-4ad2-8a54-df69f14c0f84 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:32.209317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-43c47412-8805-4008-bce5-eb9b28f68b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:32.224917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b0919-1baa-4ad2-8a54-df69f14c0f84 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:32.226573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 779/1543] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:32.235241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe25a32-5b66-4fb9-bc43-18a88176af1f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.235604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe25a32-5b66-4fb9-bc43-18a88176af1f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.235888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe25a32-5b66-4fb9-bc43-18a88176af1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.236709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6040e14-7194-4c8b-935b-918dea1b077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:32.237626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6040e14-7194-4c8b-935b-918dea1b077f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.238108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6040e14-7194-4c8b-935b-918dea1b077f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.238608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6040e14-7194-4c8b-935b-918dea1b077f admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.239508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6040e14-7194-4c8b-935b-918dea1b077f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:32.241752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6040e14-7194-4c8b-935b-918dea1b077f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:32.275395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0fe25a32-5b66-4fb9-bc43-18a88176af1f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:32.276876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0fe25a32-5b66-4fb9-bc43-18a88176af1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:32.291964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 780/1544] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:42:32.297951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe25a32-5b66-4fb9-bc43-18a88176af1f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:32.313619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe25a32-5b66-4fb9-bc43-18a88176af1f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:32.333585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8647b5-0225-44f9-8d21-86750c3b8354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:32.334233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8647b5-0225-44f9-8d21-86750c3b8354 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.334515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8647b5-0225-44f9-8d21-86750c3b8354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.334791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8647b5-0225-44f9-8d21-86750c3b8354 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.335629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8647b5-0225-44f9-8d21-86750c3b8354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:32.335753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8647b5-0225-44f9-8d21-86750c3b8354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:32.337379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8647b5-0225-44f9-8d21-86750c3b8354 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:32.357118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 781/1545] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:42:32.373794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8647b5-0225-44f9-8d21-86750c3b8354 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:32.389557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8647b5-0225-44f9-8d21-86750c3b8354 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:32.408417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cb442-d86e-4300-8a0a-2a4483240669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:32.408949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cb442-d86e-4300-8a0a-2a4483240669 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.409233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cb442-d86e-4300-8a0a-2a4483240669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.409513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cb442-d86e-4300-8a0a-2a4483240669 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.410175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cb442-d86e-4300-8a0a-2a4483240669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:32.416468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 765/1546] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/projects => generated 371 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:42:32.417874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-ea1cb442-d86e-4300-8a0a-2a4483240669 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:42:32.417874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Oct 04 05:42:32.417874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: Oct 04 05:42:32.419143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cb442-d86e-4300-8a0a-2a4483240669 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:32.428083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6040e14-7194-4c8b-935b-918dea1b077f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:32.599414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 782/1547] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/projects => generated 383 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:32.603323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b772e8-c0fc-4bd2-afce-bcda61dae046 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.603619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b772e8-c0fc-4bd2-afce-bcda61dae046 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.603900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b772e8-c0fc-4bd2-afce-bcda61dae046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.650322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6040e14-7194-4c8b-935b-918dea1b077f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:32.665524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-01b772e8-c0fc-4bd2-afce-bcda61dae046 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:32.670169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-01b772e8-c0fc-4bd2-afce-bcda61dae046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:32.811651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13068f-3277-43f8-a924-7792c3e7b5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:32.813017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13068f-3277-43f8-a924-7792c3e7b5fd admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.813653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13068f-3277-43f8-a924-7792c3e7b5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.814201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13068f-3277-43f8-a924-7792c3e7b5fd admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.815244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13068f-3277-43f8-a924-7792c3e7b5fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:32.815374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 783/1548] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:32.816575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-2d13068f-3277-43f8-a924-7792c3e7b5fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:42:32.816575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 04 05:42:32.816575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: Oct 04 05:42:32.818028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933367f4-0c4c-4b83-83d1-8292e4b6c1c6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.818402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933367f4-0c4c-4b83-83d1-8292e4b6c1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.818743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933367f4-0c4c-4b83-83d1-8292e4b6c1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.819885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13068f-3277-43f8-a924-7792c3e7b5fd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:32.850345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-933367f4-0c4c-4b83-83d1-8292e4b6c1c6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:32.854269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-933367f4-0c4c-4b83-83d1-8292e4b6c1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:32.863139 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 784/1549] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:32.865250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc33fd-c60d-46ea-b4ab-4c05abae33cd None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.865531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc33fd-c60d-46ea-b4ab-4c05abae33cd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.865802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc33fd-c60d-46ea-b4ab-4c05abae33cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.896812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-aacc33fd-c60d-46ea-b4ab-4c05abae33cd None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:32.898017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aacc33fd-c60d-46ea-b4ab-4c05abae33cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:32.906313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 785/1550] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:32.907068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc33fd-c60d-46ea-b4ab-4c05abae33cd None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:32.918277 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc33fd-c60d-46ea-b4ab-4c05abae33cd None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:32.922877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84150510-0b83-4436-8b7d-886259638986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:32.923475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84150510-0b83-4436-8b7d-886259638986 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.923758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84150510-0b83-4436-8b7d-886259638986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.924031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84150510-0b83-4436-8b7d-886259638986 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.924688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84150510-0b83-4436-8b7d-886259638986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:32.926162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 766/1551] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/users => generated 286 bytes in 499 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:32.927225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84150510-0b83-4436-8b7d-886259638986 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:32.928273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce1969-c9d8-47e3-a7b2-1c3612f09cde None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.928615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce1969-c9d8-47e3-a7b2-1c3612f09cde None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.928894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce1969-c9d8-47e3-a7b2-1c3612f09cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.948860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-adce1969-c9d8-47e3-a7b2-1c3612f09cde None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:32.950057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-adce1969-c9d8-47e3-a7b2-1c3612f09cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:32.962175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 767/1552] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:32.962175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42eff0-2bed-43aa-a151-a53f11b9d38f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.962175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42eff0-2bed-43aa-a151-a53f11b9d38f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.962175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42eff0-2bed-43aa-a151-a53f11b9d38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:32.979924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7d42eff0-2bed-43aa-a151-a53f11b9d38f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:32.987198 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7d42eff0-2bed-43aa-a151-a53f11b9d38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:32.989833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 768/1553] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:32.991005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d332bd1-1659-47b6-a5b1-a8a284ab747c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:32.991286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d332bd1-1659-47b6-a5b1-a8a284ab747c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:32.991558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d332bd1-1659-47b6-a5b1-a8a284ab747c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.011137 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6d332bd1-1659-47b6-a5b1-a8a284ab747c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:33.012325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6d332bd1-1659-47b6-a5b1-a8a284ab747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:33.015837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 786/1554] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:33.016091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84069676-5450-4cd9-a4d5-7b3b1e3908fe None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.016379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84069676-5450-4cd9-a4d5-7b3b1e3908fe None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.016645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84069676-5450-4cd9-a4d5-7b3b1e3908fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.021294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 769/1555] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:33.022681 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19c6e3-657e-47b8-92f7-dd26c4b0ad0f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.022973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19c6e3-657e-47b8-92f7-dd26c4b0ad0f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.023255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19c6e3-657e-47b8-92f7-dd26c4b0ad0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.037637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-84069676-5450-4cd9-a4d5-7b3b1e3908fe None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:33.040520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-84069676-5450-4cd9-a4d5-7b3b1e3908fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:33.048101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9f19c6e3-657e-47b8-92f7-dd26c4b0ad0f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:33.053726 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 787/1556] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:33.053850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9f19c6e3-657e-47b8-92f7-dd26c4b0ad0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:33.053980 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590546-76b1-494a-9ba8-c01218f7df5d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.053980 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590546-76b1-494a-9ba8-c01218f7df5d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.053980 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e590546-76b1-494a-9ba8-c01218f7df5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.058644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 770/1557] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:33.064294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19c6e3-657e-47b8-92f7-dd26c4b0ad0f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.072843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5e590546-76b1-494a-9ba8-c01218f7df5d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:33.074044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5e590546-76b1-494a-9ba8-c01218f7df5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:33.074193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19c6e3-657e-47b8-92f7-dd26c4b0ad0f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.079388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61419828-02d0-46cd-96df-f5e18ce7ab70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.080001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61419828-02d0-46cd-96df-f5e18ce7ab70 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.080280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61419828-02d0-46cd-96df-f5e18ce7ab70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.080554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61419828-02d0-46cd-96df-f5e18ce7ab70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.081217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61419828-02d0-46cd-96df-f5e18ce7ab70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.082292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 788/1558] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:33.082946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61419828-02d0-46cd-96df-f5e18ce7ab70 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.083489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d396e0-b0d8-4e4a-b0ac-b345a6ec0710 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.083772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d396e0-b0d8-4e4a-b0ac-b345a6ec0710 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.084043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d396e0-b0d8-4e4a-b0ac-b345a6ec0710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.093259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 771/1559] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:33.094520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96335-05f5-4fd5-a227-5c325bbb3028 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.094804 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96335-05f5-4fd5-a227-5c325bbb3028 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.095079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96335-05f5-4fd5-a227-5c325bbb3028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.106421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-19d396e0-b0d8-4e4a-b0ac-b345a6ec0710 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:33.107797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-19d396e0-b0d8-4e4a-b0ac-b345a6ec0710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:33.114834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dce96335-05f5-4fd5-a227-5c325bbb3028 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:33.116009 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dce96335-05f5-4fd5-a227-5c325bbb3028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:33.117022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 789/1560] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:33.117781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d396e0-b0d8-4e4a-b0ac-b345a6ec0710 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.124076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 772/1561] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:33.126986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96335-05f5-4fd5-a227-5c325bbb3028 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.129803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d396e0-b0d8-4e4a-b0ac-b345a6ec0710 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.131740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:33.134484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b77a3-1f7d-49b8-adc4-e9e9d6ff7faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.135084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b77a3-1f7d-49b8-adc4-e9e9d6ff7faf admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.135362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b77a3-1f7d-49b8-adc4-e9e9d6ff7faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.135635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b77a3-1f7d-49b8-adc4-e9e9d6ff7faf admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.136276 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b77a3-1f7d-49b8-adc4-e9e9d6ff7faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.137912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b77a3-1f7d-49b8-adc4-e9e9d6ff7faf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.140383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96335-05f5-4fd5-a227-5c325bbb3028 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.146566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43120e5-67f1-4f65-bb0b-9069fc411849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.147629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43120e5-67f1-4f65-bb0b-9069fc411849 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.148008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43120e5-67f1-4f65-bb0b-9069fc411849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.148122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 790/1562] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:33.148496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43120e5-67f1-4f65-bb0b-9069fc411849 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.148865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b77a3-1f7d-49b8-adc4-e9e9d6ff7faf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.149482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43120e5-67f1-4f65-bb0b-9069fc411849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:33.149838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43120e5-67f1-4f65-bb0b-9069fc411849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.152370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43120e5-67f1-4f65-bb0b-9069fc411849 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.158586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b77a3-1f7d-49b8-adc4-e9e9d6ff7faf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.164095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba8670-5058-45cf-98e4-4d2bffabb2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.164713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba8670-5058-45cf-98e4-4d2bffabb2dd admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.164990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba8670-5058-45cf-98e4-4d2bffabb2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.165262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba8670-5058-45cf-98e4-4d2bffabb2dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.165927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba8670-5058-45cf-98e4-4d2bffabb2dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:33.166265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba8670-5058-45cf-98e4-4d2bffabb2dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.173045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 773/1563] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:33.173203 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba8670-5058-45cf-98e4-4d2bffabb2dd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.173311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e50701-be77-403f-80e0-919f6e99ef75 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.173311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e50701-be77-403f-80e0-919f6e99ef75 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.173311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e50701-be77-403f-80e0-919f6e99ef75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.199256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 791/1564] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:33.199489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e6e50701-be77-403f-80e0-919f6e99ef75 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:33.199596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba8670-5058-45cf-98e4-4d2bffabb2dd admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.199692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e6e50701-be77-403f-80e0-919f6e99ef75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:33.203217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 774/1565] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:33.204071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e50701-be77-403f-80e0-919f6e99ef75 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.205891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba8670-5058-45cf-98e4-4d2bffabb2dd admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.211465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255e8a3-a9d3-42ab-a13e-b68cd545f81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.212236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255e8a3-a9d3-42ab-a13e-b68cd545f81d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.212587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255e8a3-a9d3-42ab-a13e-b68cd545f81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.212927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255e8a3-a9d3-42ab-a13e-b68cd545f81d admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.213428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e50701-be77-403f-80e0-919f6e99ef75 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.213684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255e8a3-a9d3-42ab-a13e-b68cd545f81d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:33.214181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255e8a3-a9d3-42ab-a13e-b68cd545f81d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.216235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255e8a3-a9d3-42ab-a13e-b68cd545f81d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.218131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f0605-07fe-42ad-9f64-9959e2191e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.218731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f0605-07fe-42ad-9f64-9959e2191e95 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.219011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f0605-07fe-42ad-9f64-9959e2191e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.219281 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f0605-07fe-42ad-9f64-9959e2191e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.219910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f0605-07fe-42ad-9f64-9959e2191e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.221489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f0605-07fe-42ad-9f64-9959e2191e95 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.228940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 792/1566] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:33.232116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 775/1567] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:33.233005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f0605-07fe-42ad-9f64-9959e2191e95 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.233208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255e8a3-a9d3-42ab-a13e-b68cd545f81d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.243180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f0605-07fe-42ad-9f64-9959e2191e95 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.245844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255e8a3-a9d3-42ab-a13e-b68cd545f81d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.247676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b409276-bfc0-499a-9f15-a43f4e3ae4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.248254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b409276-bfc0-499a-9f15-a43f4e3ae4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.248528 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b409276-bfc0-499a-9f15-a43f4e3ae4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.248793 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b409276-bfc0-499a-9f15-a43f4e3ae4e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.249418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b409276-bfc0-499a-9f15-a43f4e3ae4e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:33.249715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b409276-bfc0-499a-9f15-a43f4e3ae4e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.250449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcab47c-0f00-406f-85cf-28e2b6afed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.251024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcab47c-0f00-406f-85cf-28e2b6afed29 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.251302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcab47c-0f00-406f-85cf-28e2b6afed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.251407 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b409276-bfc0-499a-9f15-a43f4e3ae4e6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.251580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcab47c-0f00-406f-85cf-28e2b6afed29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.252248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcab47c-0f00-406f-85cf-28e2b6afed29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:33.252729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcab47c-0f00-406f-85cf-28e2b6afed29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.254861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcab47c-0f00-406f-85cf-28e2b6afed29 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.263059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 776/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:33.263835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b409276-bfc0-499a-9f15-a43f4e3ae4e6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.268991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 793/1569] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:33.269638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcab47c-0f00-406f-85cf-28e2b6afed29 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.273200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b409276-bfc0-499a-9f15-a43f4e3ae4e6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.276214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:33.278784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b92bb-bf02-498a-bb8c-9ab09e0454df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.279370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b92bb-bf02-498a-bb8c-9ab09e0454df admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.279644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b92bb-bf02-498a-bb8c-9ab09e0454df admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.279912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b92bb-bf02-498a-bb8c-9ab09e0454df admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.280351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcab47c-0f00-406f-85cf-28e2b6afed29 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.280538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b92bb-bf02-498a-bb8c-9ab09e0454df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.282093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b92bb-bf02-498a-bb8c-9ab09e0454df admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.284925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362c13f-3699-4349-8e6e-0f9a3fbd5808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.287247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c13f-3699-4349-8e6e-0f9a3fbd5808 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.287247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c13f-3699-4349-8e6e-0f9a3fbd5808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.287247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c13f-3699-4349-8e6e-0f9a3fbd5808 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.287518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362c13f-3699-4349-8e6e-0f9a3fbd5808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.289171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362c13f-3699-4349-8e6e-0f9a3fbd5808 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.295863 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 777/1570] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:33.295863 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b92bb-bf02-498a-bb8c-9ab09e0454df admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.304678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b92bb-bf02-498a-bb8c-9ab09e0454df admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.309541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ec475-d315-419c-b2b7-e3757dfb428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.310130 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ec475-d315-419c-b2b7-e3757dfb428a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.310441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ec475-d315-419c-b2b7-e3757dfb428a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.310715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ec475-d315-419c-b2b7-e3757dfb428a admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.311376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ec475-d315-419c-b2b7-e3757dfb428a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.313470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ec475-d315-419c-b2b7-e3757dfb428a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.364060 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 794/1571] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:33.365055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362c13f-3699-4349-8e6e-0f9a3fbd5808 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.387417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 778/1572] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:33.388170 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ec475-d315-419c-b2b7-e3757dfb428a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.432248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362c13f-3699-4349-8e6e-0f9a3fbd5808 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.452598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ec475-d315-419c-b2b7-e3757dfb428a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.458359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ce6f6-eda6-4449-9eff-1913b97c78ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.458511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ce6f6-eda6-4449-9eff-1913b97c78ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.458789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ce6f6-eda6-4449-9eff-1913b97c78ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.459056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ce6f6-eda6-4449-9eff-1913b97c78ad admin admin] PATH_INFO: `/v3/projects/845a814bddd847a99ac186b52197aa3f/users/11f15a53ee034000a29d5ed684c052db/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.474788 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ce6f6-eda6-4449-9eff-1913b97c78ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=845a814bddd847a99ac186b52197aa3f, user_id=11f15a53ee034000a29d5ed684c052db, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.477429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ce6f6-eda6-4449-9eff-1913b97c78ad admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.497324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00bbf-929d-4f6c-97c3-4ee1dfbbeaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.497958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00bbf-929d-4f6c-97c3-4ee1dfbbeaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.498268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00bbf-929d-4f6c-97c3-4ee1dfbbeaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.498537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b00bbf-929d-4f6c-97c3-4ee1dfbbeaa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.499171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b00bbf-929d-4f6c-97c3-4ee1dfbbeaa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:33.499469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b00bbf-929d-4f6c-97c3-4ee1dfbbeaa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.501044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b00bbf-929d-4f6c-97c3-4ee1dfbbeaa1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.514992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 795/1573] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:33.515841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00bbf-929d-4f6c-97c3-4ee1dfbbeaa1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.527268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b00bbf-929d-4f6c-97c3-4ee1dfbbeaa1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.532143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7536e-00ac-4f0b-9ede-9cd88d107a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.532730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7536e-00ac-4f0b-9ede-9cd88d107a82 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.533012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7536e-00ac-4f0b-9ede-9cd88d107a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.533312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7536e-00ac-4f0b-9ede-9cd88d107a82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.533966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7536e-00ac-4f0b-9ede-9cd88d107a82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.536608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7536e-00ac-4f0b-9ede-9cd88d107a82 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.569446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 779/1574] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:33 2022] PUT /identity/v3/projects/845a814bddd847a99ac186b52197aa3f/users/11f15a53ee034000a29d5ed684c052db/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:33.570285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ce6f6-eda6-4449-9eff-1913b97c78ad admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.631229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ce6f6-eda6-4449-9eff-1913b97c78ad admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.633403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 796/1575] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:33.634426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7536e-00ac-4f0b-9ede-9cd88d107a82 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.643161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:33.689943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3a3eb-e86d-4a26-ae9b-c7ea6053c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.690621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a3eb-e86d-4a26-ae9b-c7ea6053c23a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.690903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a3eb-e86d-4a26-ae9b-c7ea6053c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.691178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a3eb-e86d-4a26-ae9b-c7ea6053c23a admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.691808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3a3eb-e86d-4a26-ae9b-c7ea6053c23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.694241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3a3eb-e86d-4a26-ae9b-c7ea6053c23a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.698752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7536e-00ac-4f0b-9ede-9cd88d107a82 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.703338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1f5f0-7d20-49ea-a80e-e03168e514e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.704067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1f5f0-7d20-49ea-a80e-e03168e514e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.704363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1f5f0-7d20-49ea-a80e-e03168e514e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.704641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1f5f0-7d20-49ea-a80e-e03168e514e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.705275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1f5f0-7d20-49ea-a80e-e03168e514e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.707210 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1f5f0-7d20-49ea-a80e-e03168e514e0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.714122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:33.787219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 780/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/projects => generated 405 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:33.788226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3a3eb-e86d-4a26-ae9b-c7ea6053c23a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.820470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 797/1577] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:33.821582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1f5f0-7d20-49ea-a80e-e03168e514e0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:33.851698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3a3eb-e86d-4a26-ae9b-c7ea6053c23a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.882923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1f5f0-7d20-49ea-a80e-e03168e514e0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:33.892448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f4ea6-9764-4d6f-a151-1c4279354f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.893138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4ea6-9764-4d6f-a151-1c4279354f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.893492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4ea6-9764-4d6f-a151-1c4279354f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.894518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4ea6-9764-4d6f-a151-1c4279354f14 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.895820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f4ea6-9764-4d6f-a151-1c4279354f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.897805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f4ea6-9764-4d6f-a151-1c4279354f14 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:33.910720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abae3f9-5769-40cc-ad7c-ddd328e79057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:33.911485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abae3f9-5769-40cc-ad7c-ddd328e79057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:33.911724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abae3f9-5769-40cc-ad7c-ddd328e79057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:33.911996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abae3f9-5769-40cc-ad7c-ddd328e79057 admin admin] PATH_INFO: `/v3/projects/f7b850d4b5df4a4e93720cb855add44f/users/c66c7ddbf18d4f1fb8c890f548031f40/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:33.926133 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abae3f9-5769-40cc-ad7c-ddd328e79057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b850d4b5df4a4e93720cb855add44f, user_id=c66c7ddbf18d4f1fb8c890f548031f40, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:33.928885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abae3f9-5769-40cc-ad7c-ddd328e79057 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.017816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 798/1578] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:33 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:34.018586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f4ea6-9764-4d6f-a151-1c4279354f14 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.028826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f4ea6-9764-4d6f-a151-1c4279354f14 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.033586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b16d2-adf4-4847-90ca-1d6a9c47a4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.034190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b16d2-adf4-4847-90ca-1d6a9c47a4f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.034465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b16d2-adf4-4847-90ca-1d6a9c47a4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.037801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b16d2-adf4-4847-90ca-1d6a9c47a4f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.037801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b16d2-adf4-4847-90ca-1d6a9c47a4f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.037801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b16d2-adf4-4847-90ca-1d6a9c47a4f8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.038087 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 781/1579] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:33 2022] PUT /identity/v3/projects/f7b850d4b5df4a4e93720cb855add44f/users/c66c7ddbf18d4f1fb8c890f548031f40/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:34.038535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abae3f9-5769-40cc-ad7c-ddd328e79057 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.075290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:34.102294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abae3f9-5769-40cc-ad7c-ddd328e79057 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.106909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0aaccc-966b-4b29-9ddc-3a6a2ccd6420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.107562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aaccc-966b-4b29-9ddc-3a6a2ccd6420 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.107883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aaccc-966b-4b29-9ddc-3a6a2ccd6420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.108155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aaccc-966b-4b29-9ddc-3a6a2ccd6420 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.108770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0aaccc-966b-4b29-9ddc-3a6a2ccd6420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.110724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0aaccc-966b-4b29-9ddc-3a6a2ccd6420 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.125992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 799/1580] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:34 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:34.127706 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68d5a-fb03-407f-a1c4-63fd3bce16d1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.128080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68d5a-fb03-407f-a1c4-63fd3bce16d1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.128460 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68d5a-fb03-407f-a1c4-63fd3bce16d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.155662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d4f68d5a-fb03-407f-a1c4-63fd3bce16d1 None None] MFA Rules not processed for user `11f15a53ee034000a29d5ed684c052db`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:34.156986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d4f68d5a-fb03-407f-a1c4-63fd3bce16d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:34.209981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 782/1581] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:34 2022] POST /identity/v3/projects => generated 403 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:34.213987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0aaccc-966b-4b29-9ddc-3a6a2ccd6420 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.221338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 800/1582] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:34.222240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f68d5a-fb03-407f-a1c4-63fd3bce16d1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.233255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f68d5a-fb03-407f-a1c4-63fd3bce16d1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.277446 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:34.278161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0aaccc-966b-4b29-9ddc-3a6a2ccd6420 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.282784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97297d-eec4-4dcb-a14a-98d415672259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.283394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97297d-eec4-4dcb-a14a-98d415672259 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.283669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97297d-eec4-4dcb-a14a-98d415672259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.283942 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97297d-eec4-4dcb-a14a-98d415672259 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.284566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97297d-eec4-4dcb-a14a-98d415672259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.286558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97297d-eec4-4dcb-a14a-98d415672259 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.289061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2d772-38af-4b8d-8e4a-d5bbf4989d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.289436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2d772-38af-4b8d-8e4a-d5bbf4989d29 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.289713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2d772-38af-4b8d-8e4a-d5bbf4989d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.289985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2d772-38af-4b8d-8e4a-d5bbf4989d29 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.290643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2d772-38af-4b8d-8e4a-d5bbf4989d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.292564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2d772-38af-4b8d-8e4a-d5bbf4989d29 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.388947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 783/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:34 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:34.389969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97297d-eec4-4dcb-a14a-98d415672259 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.399764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97297d-eec4-4dcb-a14a-98d415672259 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.404419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4ac23-954c-49ce-8586-d3fb9f79e8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.404990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4ac23-954c-49ce-8586-d3fb9f79e8ab admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.405268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4ac23-954c-49ce-8586-d3fb9f79e8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.405538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4ac23-954c-49ce-8586-d3fb9f79e8ab admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.406167 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4ac23-954c-49ce-8586-d3fb9f79e8ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.408224 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4ac23-954c-49ce-8586-d3fb9f79e8ab admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.417020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 801/1584] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:34 2022] POST /identity/v3/users => generated 303 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:34.417020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2d772-38af-4b8d-8e4a-d5bbf4989d29 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.426403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2d772-38af-4b8d-8e4a-d5bbf4989d29 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.431119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c36399-ab4a-4b56-93e1-c9df3509d8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.431704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c36399-ab4a-4b56-93e1-c9df3509d8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.431978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c36399-ab4a-4b56-93e1-c9df3509d8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.432245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c36399-ab4a-4b56-93e1-c9df3509d8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.432898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c36399-ab4a-4b56-93e1-c9df3509d8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.434503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c36399-ab4a-4b56-93e1-c9df3509d8b4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.444824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 802/1585] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:34.445986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fd6e6-ee23-4cc2-82af-13ccc9c93162 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.447029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fd6e6-ee23-4cc2-82af-13ccc9c93162 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.447029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fd6e6-ee23-4cc2-82af-13ccc9c93162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.464801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-738fd6e6-ee23-4cc2-82af-13ccc9c93162 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:34.465977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-738fd6e6-ee23-4cc2-82af-13ccc9c93162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:34.473508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 803/1586] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:34.479663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fd6e6-ee23-4cc2-82af-13ccc9c93162 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.483500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fd6e6-ee23-4cc2-82af-13ccc9c93162 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.488390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125557f-6302-4d57-89e3-746dfb83314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.489074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125557f-6302-4d57-89e3-746dfb83314c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.489415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125557f-6302-4d57-89e3-746dfb83314c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.489691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125557f-6302-4d57-89e3-746dfb83314c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.490370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125557f-6302-4d57-89e3-746dfb83314c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.491959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125557f-6302-4d57-89e3-746dfb83314c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.501834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 804/1587] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:34.502741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125557f-6302-4d57-89e3-746dfb83314c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.503956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 784/1588] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:34 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:34.505292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4ac23-954c-49ce-8586-d3fb9f79e8ab admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.514932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4ac23-954c-49ce-8586-d3fb9f79e8ab admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.516819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125557f-6302-4d57-89e3-746dfb83314c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.519625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72453a-835e-4ef5-895a-9904ce212a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.520327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72453a-835e-4ef5-895a-9904ce212a16 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.520606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72453a-835e-4ef5-895a-9904ce212a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.520873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72453a-835e-4ef5-895a-9904ce212a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.521489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72453a-835e-4ef5-895a-9904ce212a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.521700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fcd39-8ae9-4a9e-8062-4f74eb52026e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.522329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fcd39-8ae9-4a9e-8062-4f74eb52026e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.522607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fcd39-8ae9-4a9e-8062-4f74eb52026e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.522878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fcd39-8ae9-4a9e-8062-4f74eb52026e admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.523415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72453a-835e-4ef5-895a-9904ce212a16 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.523518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77fcd39-8ae9-4a9e-8062-4f74eb52026e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.525477 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77fcd39-8ae9-4a9e-8062-4f74eb52026e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.605796 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 805/1589] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:34 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:34.609031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fcd39-8ae9-4a9e-8062-4f74eb52026e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.616838 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fcd39-8ae9-4a9e-8062-4f74eb52026e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.621400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f816d-ca12-43a2-983b-c00a00a0466d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.622014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f816d-ca12-43a2-983b-c00a00a0466d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.622327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f816d-ca12-43a2-983b-c00a00a0466d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.622604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f816d-ca12-43a2-983b-c00a00a0466d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.623233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f816d-ca12-43a2-983b-c00a00a0466d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.624819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f816d-ca12-43a2-983b-c00a00a0466d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.630453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 785/1590] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:34 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:34.634643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72453a-835e-4ef5-895a-9904ce212a16 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.638205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 806/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:34.639091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f816d-ca12-43a2-983b-c00a00a0466d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.648972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f816d-ca12-43a2-983b-c00a00a0466d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.694608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72453a-835e-4ef5-895a-9904ce212a16 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.699323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e20ea7-9dfc-48ad-a201-518da3378d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.699932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e20ea7-9dfc-48ad-a201-518da3378d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.700215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e20ea7-9dfc-48ad-a201-518da3378d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.700492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e20ea7-9dfc-48ad-a201-518da3378d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.701139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e20ea7-9dfc-48ad-a201-518da3378d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.702841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e20ea7-9dfc-48ad-a201-518da3378d0e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.707415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129de572-db5f-47aa-bd45-a47684616798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.708472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129de572-db5f-47aa-bd45-a47684616798 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.708756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129de572-db5f-47aa-bd45-a47684616798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.709030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129de572-db5f-47aa-bd45-a47684616798 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.709668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129de572-db5f-47aa-bd45-a47684616798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.711288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129de572-db5f-47aa-bd45-a47684616798 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.712779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 786/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:34.713591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e20ea7-9dfc-48ad-a201-518da3378d0e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.717764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:34.721456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 807/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:34.722353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129de572-db5f-47aa-bd45-a47684616798 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.733374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129de572-db5f-47aa-bd45-a47684616798 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.738235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787983f-73d8-448f-b9a1-58625b18b24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.739514 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787983f-73d8-448f-b9a1-58625b18b24b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.739787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787983f-73d8-448f-b9a1-58625b18b24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.740056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787983f-73d8-448f-b9a1-58625b18b24b admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.740679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787983f-73d8-448f-b9a1-58625b18b24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.742283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787983f-73d8-448f-b9a1-58625b18b24b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.752931 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 808/1594] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:34.753828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787983f-73d8-448f-b9a1-58625b18b24b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.764399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787983f-73d8-448f-b9a1-58625b18b24b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.769130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdef6d2-3f22-4413-8dc6-737104477a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.769714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdef6d2-3f22-4413-8dc6-737104477a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.770033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdef6d2-3f22-4413-8dc6-737104477a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.770352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdef6d2-3f22-4413-8dc6-737104477a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.770739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e20ea7-9dfc-48ad-a201-518da3378d0e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.770983 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdef6d2-3f22-4413-8dc6-737104477a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.772601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdef6d2-3f22-4413-8dc6-737104477a2e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.774358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08f96c-46cd-4e72-a8c8-4926e8240cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.774955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08f96c-46cd-4e72-a8c8-4926e8240cef service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.775226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08f96c-46cd-4e72-a8c8-4926e8240cef service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.775498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08f96c-46cd-4e72-a8c8-4926e8240cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.786483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 809/1595] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:34.786483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdef6d2-3f22-4413-8dc6-737104477a2e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.786738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08f96c-46cd-4e72-a8c8-4926e8240cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.787080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08f96c-46cd-4e72-a8c8-4926e8240cef service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.795504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdef6d2-3f22-4413-8dc6-737104477a2e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.797422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 787/1596] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:34.798307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08f96c-46cd-4e72-a8c8-4926e8240cef service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.800426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157e7f9-37cc-41a8-b14a-6fecd72c755f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.801017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157e7f9-37cc-41a8-b14a-6fecd72c755f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.801862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157e7f9-37cc-41a8-b14a-6fecd72c755f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.802169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157e7f9-37cc-41a8-b14a-6fecd72c755f admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.802800 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5157e7f9-37cc-41a8-b14a-6fecd72c755f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.804379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5157e7f9-37cc-41a8-b14a-6fecd72c755f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.808559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08f96c-46cd-4e72-a8c8-4926e8240cef service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.814407 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba85252-d8af-456e-9089-721b85b200ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.817810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 810/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:34.817953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba85252-d8af-456e-9089-721b85b200ad admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.817953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba85252-d8af-456e-9089-721b85b200ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.818119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157e7f9-37cc-41a8-b14a-6fecd72c755f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.818246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba85252-d8af-456e-9089-721b85b200ad admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.818246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba85252-d8af-456e-9089-721b85b200ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.818395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba85252-d8af-456e-9089-721b85b200ad admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.828578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157e7f9-37cc-41a8-b14a-6fecd72c755f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.834112 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daf0a1-f24b-4542-9b0d-53231ba9c1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.834745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daf0a1-f24b-4542-9b0d-53231ba9c1ba admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.835025 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daf0a1-f24b-4542-9b0d-53231ba9c1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.835294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daf0a1-f24b-4542-9b0d-53231ba9c1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.835946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daf0a1-f24b-4542-9b0d-53231ba9c1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.837579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daf0a1-f24b-4542-9b0d-53231ba9c1ba admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.847683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 811/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:34.849339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daf0a1-f24b-4542-9b0d-53231ba9c1ba admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.858705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daf0a1-f24b-4542-9b0d-53231ba9c1ba admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.863219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1eee2-aab9-4778-99f9-f1cd2daec504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.868761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1eee2-aab9-4778-99f9-f1cd2daec504 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.868761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1eee2-aab9-4778-99f9-f1cd2daec504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.868761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1eee2-aab9-4778-99f9-f1cd2daec504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.868761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1eee2-aab9-4778-99f9-f1cd2daec504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.868761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1eee2-aab9-4778-99f9-f1cd2daec504 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.877398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 812/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:34.878326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1eee2-aab9-4778-99f9-f1cd2daec504 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.888375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1eee2-aab9-4778-99f9-f1cd2daec504 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.893372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5789c4b-24f2-41ea-8f53-8f89f3cdf8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.894043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5789c4b-24f2-41ea-8f53-8f89f3cdf8c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.895459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5789c4b-24f2-41ea-8f53-8f89f3cdf8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.895744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5789c4b-24f2-41ea-8f53-8f89f3cdf8c8 admin admin] PATH_INFO: `/v3/projects/b427232479e14a6fa576f4461032f12b/users/9d1d5de04c894a2ab780bcdee15f5c5b/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.905427 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 788/1600] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:34 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:34.906516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba85252-d8af-456e-9089-721b85b200ad admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:34.917096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5789c4b-24f2-41ea-8f53-8f89f3cdf8c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b427232479e14a6fa576f4461032f12b, user_id=9d1d5de04c894a2ab780bcdee15f5c5b, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.917354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba85252-d8af-456e-9089-721b85b200ad admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:34.918322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-c5789c4b-24f2-41ea-8f53-8f89f3cdf8c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:42:34.918322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Oct 04 05:42:34.918322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: Oct 04 05:42:34.920290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5789c4b-24f2-41ea-8f53-8f89f3cdf8c8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:34.922303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c848edc-2913-4a9c-ba67-14b861fb2061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:34.923061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c848edc-2913-4a9c-ba67-14b861fb2061 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:34.923271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c848edc-2913-4a9c-ba67-14b861fb2061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:34.923589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c848edc-2913-4a9c-ba67-14b861fb2061 admin admin] PATH_INFO: `/v3/projects/7070376ed1fd4d289d306c65471b5e95/users/cd8486004f1141218103f1add322fca9/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:34.940239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c848edc-2913-4a9c-ba67-14b861fb2061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7070376ed1fd4d289d306c65471b5e95, user_id=cd8486004f1141218103f1add322fca9, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:34.943477 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c848edc-2913-4a9c-ba67-14b861fb2061 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:35.004057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:35.009823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 813/1601] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:34 2022] PUT /identity/v3/projects/b427232479e14a6fa576f4461032f12b/users/9d1d5de04c894a2ab780bcdee15f5c5b/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:35.010756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5789c4b-24f2-41ea-8f53-8f89f3cdf8c8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:35.038618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 789/1602] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:34 2022] PUT /identity/v3/projects/7070376ed1fd4d289d306c65471b5e95/users/cd8486004f1141218103f1add322fca9/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:35.041060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c848edc-2913-4a9c-ba67-14b861fb2061 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:35.084486 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5789c4b-24f2-41ea-8f53-8f89f3cdf8c8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:35.121068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c848edc-2913-4a9c-ba67-14b861fb2061 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:35.129011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ca23a-c463-4e88-9ee0-dd3f8a17cf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:35.129726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ca23a-c463-4e88-9ee0-dd3f8a17cf77 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.130039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ca23a-c463-4e88-9ee0-dd3f8a17cf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.130347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ca23a-c463-4e88-9ee0-dd3f8a17cf77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.131034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ca23a-c463-4e88-9ee0-dd3f8a17cf77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:35.132573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ca23a-c463-4e88-9ee0-dd3f8a17cf77 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:35.143612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 790/1603] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:35.144406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ca23a-c463-4e88-9ee0-dd3f8a17cf77 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:35.155569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ca23a-c463-4e88-9ee0-dd3f8a17cf77 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:35.162653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703c22e-593b-401d-a708-9df70a33eedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:35.166235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703c22e-593b-401d-a708-9df70a33eedc admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.166235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703c22e-593b-401d-a708-9df70a33eedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.166235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703c22e-593b-401d-a708-9df70a33eedc admin admin] PATH_INFO: `/v3/projects/636e6b38bf1d4736a38728488881e9d2/users/6f1635c9c6cf4fb092c0f03a02e36174/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.167419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26ae47-138d-4981-a74b-1b01f8dab631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:35.168047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26ae47-138d-4981-a74b-1b01f8dab631 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.168328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26ae47-138d-4981-a74b-1b01f8dab631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.168603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26ae47-138d-4981-a74b-1b01f8dab631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.169334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26ae47-138d-4981-a74b-1b01f8dab631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:35.170965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26ae47-138d-4981-a74b-1b01f8dab631 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:35.182822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703c22e-593b-401d-a708-9df70a33eedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=636e6b38bf1d4736a38728488881e9d2, user_id=6f1635c9c6cf4fb092c0f03a02e36174, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:35.183437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 814/1604] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:35.184399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26ae47-138d-4981-a74b-1b01f8dab631 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:35.185771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703c22e-593b-401d-a708-9df70a33eedc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:35.196169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26ae47-138d-4981-a74b-1b01f8dab631 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:35.201541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043ac0c-27fc-4411-83b8-fbaccaa0ecac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:35.202288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ac0c-27fc-4411-83b8-fbaccaa0ecac admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.202567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ac0c-27fc-4411-83b8-fbaccaa0ecac admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.202842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043ac0c-27fc-4411-83b8-fbaccaa0ecac admin admin] PATH_INFO: `/v3/projects/1029164654a74b7ea9ae5854836a18d4/users/b60d28e09b304b1a9a431d58f55373ee/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.218534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043ac0c-27fc-4411-83b8-fbaccaa0ecac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1029164654a74b7ea9ae5854836a18d4, user_id=b60d28e09b304b1a9a431d58f55373ee, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:35.221227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043ac0c-27fc-4411-83b8-fbaccaa0ecac admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:35.283442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 791/1605] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:35 2022] PUT /identity/v3/projects/636e6b38bf1d4736a38728488881e9d2/users/6f1635c9c6cf4fb092c0f03a02e36174/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:35.284284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703c22e-593b-401d-a708-9df70a33eedc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:35.309554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 815/1606] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:35 2022] PUT /identity/v3/projects/1029164654a74b7ea9ae5854836a18d4/users/b60d28e09b304b1a9a431d58f55373ee/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:35.318458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4c7da-715f-4dba-ab39-84f1aeac56c6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.318794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4c7da-715f-4dba-ab39-84f1aeac56c6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.319242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4c7da-715f-4dba-ab39-84f1aeac56c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.345098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c0e4c7da-715f-4dba-ab39-84f1aeac56c6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:35.346359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c0e4c7da-715f-4dba-ab39-84f1aeac56c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:35.353495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:35.379168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703c22e-593b-401d-a708-9df70a33eedc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:35.426572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 816/1607] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:35.430613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa61c-37ef-45f5-b03c-663f0c92a3ec None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.430613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa61c-37ef-45f5-b03c-663f0c92a3ec None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.430613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa61c-37ef-45f5-b03c-663f0c92a3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.452881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4391ab1-58b4-45f3-9725-e8fd42cccca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:35.453618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4391ab1-58b4-45f3-9725-e8fd42cccca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.453837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4391ab1-58b4-45f3-9725-e8fd42cccca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.454108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4391ab1-58b4-45f3-9725-e8fd42cccca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.454829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4391ab1-58b4-45f3-9725-e8fd42cccca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:35.456426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4391ab1-58b4-45f3-9725-e8fd42cccca6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:35.461517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c98fa61c-37ef-45f5-b03c-663f0c92a3ec None None] MFA Rules not processed for user `cd8486004f1141218103f1add322fca9`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:35.462890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c98fa61c-37ef-45f5-b03c-663f0c92a3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:35.469233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:35.470245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 792/1608] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:35.471190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4391ab1-58b4-45f3-9725-e8fd42cccca6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:35.480899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4391ab1-58b4-45f3-9725-e8fd42cccca6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:35.485990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3fc5c-fbfa-44d9-b250-bce6b1e6dda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:35.486683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3fc5c-fbfa-44d9-b250-bce6b1e6dda2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.487013 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3fc5c-fbfa-44d9-b250-bce6b1e6dda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.487345 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3fc5c-fbfa-44d9-b250-bce6b1e6dda2 admin admin] PATH_INFO: `/v3/projects/eef5caf8a6d24a65a5758762c5bdfc3a/users/49b61dda6e4a43ca8587e7d12d4ade2e/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.505655 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3fc5c-fbfa-44d9-b250-bce6b1e6dda2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef5caf8a6d24a65a5758762c5bdfc3a, user_id=49b61dda6e4a43ca8587e7d12d4ade2e, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:35.508508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3fc5c-fbfa-44d9-b250-bce6b1e6dda2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:35.532590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 817/1609] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:35.533833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa61c-37ef-45f5-b03c-663f0c92a3ec None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:35.547389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa61c-37ef-45f5-b03c-663f0c92a3ec None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:35.552580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e638c2-429d-4c62-8b20-92fbd2d7be15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:35.553246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e638c2-429d-4c62-8b20-92fbd2d7be15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.553532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e638c2-429d-4c62-8b20-92fbd2d7be15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.553806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e638c2-429d-4c62-8b20-92fbd2d7be15 admin admin] PATH_INFO: `/v3/projects/7efd5ee13f094656992497528d761f2c/users/537981b1b3a14cb4b10f336ef0362358/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.570671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e638c2-429d-4c62-8b20-92fbd2d7be15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7efd5ee13f094656992497528d761f2c, user_id=537981b1b3a14cb4b10f336ef0362358, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:35.570898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 793/1610] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:35 2022] PUT /identity/v3/projects/eef5caf8a6d24a65a5758762c5bdfc3a/users/49b61dda6e4a43ca8587e7d12d4ade2e/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:35.572308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af362db-603f-443a-88b4-131682957c40 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.572694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af362db-603f-443a-88b4-131682957c40 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.573113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af362db-603f-443a-88b4-131682957c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.573375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e638c2-429d-4c62-8b20-92fbd2d7be15 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:35.593115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5af362db-603f-443a-88b4-131682957c40 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:35.596356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5af362db-603f-443a-88b4-131682957c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:35.649912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 818/1611] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:35 2022] PUT /identity/v3/projects/7efd5ee13f094656992497528d761f2c/users/537981b1b3a14cb4b10f336ef0362358/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:35.652511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d168c-dece-49c3-bae2-950695992cf3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.652973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d168c-dece-49c3-bae2-950695992cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.653389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d168c-dece-49c3-bae2-950695992cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.656374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 794/1612] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:35.657438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af362db-603f-443a-88b4-131682957c40 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:35.658802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5af362db-603f-443a-88b4-131682957c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:35.683982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a56d168c-dece-49c3-bae2-950695992cf3 None None] MFA Rules not processed for user `b60d28e09b304b1a9a431d58f55373ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:35.685222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a56d168c-dece-49c3-bae2-950695992cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:35.719387 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af362db-603f-443a-88b4-131682957c40 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:35.730200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:35.757560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 819/1613] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:35 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:35.757560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d168c-dece-49c3-bae2-950695992cf3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:35.779558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c1073-15a2-4e6f-99d5-9096f03b5b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:35.780473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c1073-15a2-4e6f-99d5-9096f03b5b35 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.780857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c1073-15a2-4e6f-99d5-9096f03b5b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.781237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c1073-15a2-4e6f-99d5-9096f03b5b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.833822 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d168c-dece-49c3-bae2-950695992cf3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:35.839377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe98f82-6637-4008-ac41-51c270424936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:35.840040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe98f82-6637-4008-ac41-51c270424936 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.840317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe98f82-6637-4008-ac41-51c270424936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.840587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe98f82-6637-4008-ac41-51c270424936 admin admin] PATH_INFO: `/v3/projects/e66884ef88e94c9e832ba74f0d41f381/users/a0bfec7c8a4b4b3e8d811cac8eb07fae/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.856427 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe98f82-6637-4008-ac41-51c270424936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66884ef88e94c9e832ba74f0d41f381, user_id=a0bfec7c8a4b4b3e8d811cac8eb07fae, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:35.857334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c1073-15a2-4e6f-99d5-9096f03b5b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:35.858804 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c1073-15a2-4e6f-99d5-9096f03b5b35 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:35.859131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe98f82-6637-4008-ac41-51c270424936 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:35.862239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:35.869744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 795/1614] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:35.870590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c1073-15a2-4e6f-99d5-9096f03b5b35 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:35.930309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c1073-15a2-4e6f-99d5-9096f03b5b35 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:35.934071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3c22a-9a96-4e41-af25-24d1709c4093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:35.934707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3c22a-9a96-4e41-af25-24d1709c4093 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.937538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3c22a-9a96-4e41-af25-24d1709c4093 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.937538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3c22a-9a96-4e41-af25-24d1709c4093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.938971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 820/1615] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:35 2022] PUT /identity/v3/projects/e66884ef88e94c9e832ba74f0d41f381/users/a0bfec7c8a4b4b3e8d811cac8eb07fae/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:35.940462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3fb76-261c-4666-8366-01094fcd2e95 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:35.941243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3fb76-261c-4666-8366-01094fcd2e95 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:35.941500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3fb76-261c-4666-8366-01094fcd2e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:35.959826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-45a3fb76-261c-4666-8366-01094fcd2e95 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:35.961018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-45a3fb76-261c-4666-8366-01094fcd2e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:35.996192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3c22a-9a96-4e41-af25-24d1709c4093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:35.999964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3c22a-9a96-4e41-af25-24d1709c4093 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.018314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 796/1616] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:36.020379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2529b-ddcc-4b40-8e95-1e7639db8cd1 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.020959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2529b-ddcc-4b40-8e95-1e7639db8cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.021458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2529b-ddcc-4b40-8e95-1e7639db8cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.025777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 821/1617] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:36.026933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3fb76-261c-4666-8366-01094fcd2e95 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.064316 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-86c2529b-ddcc-4b40-8e95-1e7639db8cd1 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:36.066259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-86c2529b-ddcc-4b40-8e95-1e7639db8cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:36.076294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 797/1618] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:36.079706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2529b-ddcc-4b40-8e95-1e7639db8cd1 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.094421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3fb76-261c-4666-8366-01094fcd2e95 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.097594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2529b-ddcc-4b40-8e95-1e7639db8cd1 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.099331 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbd669-9434-4da5-a780-fad3000d0c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:36.100384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbd669-9434-4da5-a780-fad3000d0c94 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.100772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbd669-9434-4da5-a780-fad3000d0c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.101232 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbd669-9434-4da5-a780-fad3000d0c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.103490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f869893-0f94-47a0-b5d8-91d9c4add25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:36.104248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f869893-0f94-47a0-b5d8-91d9c4add25d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.104627 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f869893-0f94-47a0-b5d8-91d9c4add25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.105149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f869893-0f94-47a0-b5d8-91d9c4add25d admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.105896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f869893-0f94-47a0-b5d8-91d9c4add25d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:36.108181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f869893-0f94-47a0-b5d8-91d9c4add25d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.113401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbd669-9434-4da5-a780-fad3000d0c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:36.118244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbd669-9434-4da5-a780-fad3000d0c94 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.118423 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:36.129508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 822/1619] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:36.130556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbd669-9434-4da5-a780-fad3000d0c94 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.146719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbd669-9434-4da5-a780-fad3000d0c94 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.152485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86cd30-2134-45ba-bbfc-5170be376389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:36.153107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86cd30-2134-45ba-bbfc-5170be376389 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.153442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86cd30-2134-45ba-bbfc-5170be376389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.153666 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86cd30-2134-45ba-bbfc-5170be376389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.154330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86cd30-2134-45ba-bbfc-5170be376389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:36.156014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86cd30-2134-45ba-bbfc-5170be376389 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.170260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 823/1620] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:36.170260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86cd30-2134-45ba-bbfc-5170be376389 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.180519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86cd30-2134-45ba-bbfc-5170be376389 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.184252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bb381-0885-451a-b83d-d0f053baa976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:36.184796 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bb381-0885-451a-b83d-d0f053baa976 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.185097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bb381-0885-451a-b83d-d0f053baa976 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.185377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bb381-0885-451a-b83d-d0f053baa976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.212700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 798/1621] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:36 2022] POST /identity/v3/projects => generated 405 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:36.212700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f869893-0f94-47a0-b5d8-91d9c4add25d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.214684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bb381-0885-451a-b83d-d0f053baa976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:36.216143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bb381-0885-451a-b83d-d0f053baa976 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.248439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 824/1622] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:36.249277 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bb381-0885-451a-b83d-d0f053baa976 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.264011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bb381-0885-451a-b83d-d0f053baa976 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.364534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f869893-0f94-47a0-b5d8-91d9c4add25d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.380203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1501c-4b93-4712-bc9f-b6902db000c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:36.380952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1501c-4b93-4712-bc9f-b6902db000c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.381233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1501c-4b93-4712-bc9f-b6902db000c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.381508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1501c-4b93-4712-bc9f-b6902db000c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.385361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862124b0-502d-4e69-b4fb-8503b6da0276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:36.386175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862124b0-502d-4e69-b4fb-8503b6da0276 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.386459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862124b0-502d-4e69-b4fb-8503b6da0276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.386827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862124b0-502d-4e69-b4fb-8503b6da0276 admin admin] PATH_INFO: `/v3/projects/e66884ef88e94c9e832ba74f0d41f381/users/a0bfec7c8a4b4b3e8d811cac8eb07fae/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.390430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862124b0-502d-4e69-b4fb-8503b6da0276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66884ef88e94c9e832ba74f0d41f381, user_id=a0bfec7c8a4b4b3e8d811cac8eb07fae, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:36.394472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862124b0-502d-4e69-b4fb-8503b6da0276 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.398328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1501c-4b93-4712-bc9f-b6902db000c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:36.408635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1501c-4b93-4712-bc9f-b6902db000c0 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.412708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 825/1623] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:36 2022] PUT /identity/v3/projects/e66884ef88e94c9e832ba74f0d41f381/users/a0bfec7c8a4b4b3e8d811cac8eb07fae/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:36.413992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862124b0-502d-4e69-b4fb-8503b6da0276 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.429946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:36.431415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 799/1624] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:36.432685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fb2c6-71f1-4e88-95e5-9297d94237d4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.436158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fb2c6-71f1-4e88-95e5-9297d94237d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.436158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fb2c6-71f1-4e88-95e5-9297d94237d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.485795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ac7fb2c6-71f1-4e88-95e5-9297d94237d4 None None] MFA Rules not processed for user `a0bfec7c8a4b4b3e8d811cac8eb07fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:36.491145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ac7fb2c6-71f1-4e88-95e5-9297d94237d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:36.518707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862124b0-502d-4e69-b4fb-8503b6da0276 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.526384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddf559-52bc-4602-a0d5-42130e81c8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:36.529806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddf559-52bc-4602-a0d5-42130e81c8fe admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.530181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddf559-52bc-4602-a0d5-42130e81c8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.532481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddf559-52bc-4602-a0d5-42130e81c8fe admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.533269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddf559-52bc-4602-a0d5-42130e81c8fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:36.539966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddf559-52bc-4602-a0d5-42130e81c8fe admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.613121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 800/1625] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:36.623500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fb2c6-71f1-4e88-95e5-9297d94237d4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.686218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 826/1626] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:36 2022] POST /identity/v3/users => generated 309 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:36.689491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddf559-52bc-4602-a0d5-42130e81c8fe admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.708406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddf559-52bc-4602-a0d5-42130e81c8fe admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.722072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e36f75-f4b2-4a64-be58-bc632275812a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:36.723208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e36f75-f4b2-4a64-be58-bc632275812a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.724710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e36f75-f4b2-4a64-be58-bc632275812a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.724995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e36f75-f4b2-4a64-be58-bc632275812a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.726029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e36f75-f4b2-4a64-be58-bc632275812a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:36.730860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e36f75-f4b2-4a64-be58-bc632275812a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.736335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fb2c6-71f1-4e88-95e5-9297d94237d4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.749969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb00e16-0939-440f-9549-6e3798b75d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:36.750922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb00e16-0939-440f-9549-6e3798b75d6a service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.751197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb00e16-0939-440f-9549-6e3798b75d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.751467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb00e16-0939-440f-9549-6e3798b75d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.758259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 827/1627] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:36.761868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e36f75-f4b2-4a64-be58-bc632275812a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.769028 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb00e16-0939-440f-9549-6e3798b75d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:36.772024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb00e16-0939-440f-9549-6e3798b75d6a service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.775503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e36f75-f4b2-4a64-be58-bc632275812a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.781888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2d7b1-ed6c-44f4-a74c-d498bf82fd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:36.782712 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2d7b1-ed6c-44f4-a74c-d498bf82fd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:36.783122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2d7b1-ed6c-44f4-a74c-d498bf82fd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:36.783401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2d7b1-ed6c-44f4-a74c-d498bf82fd2f admin admin] PATH_INFO: `/v3/projects/8fc523b011464612b996bad60b7e7e6e/users/27abbdf1560341b8b245ca9e7c29e524/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:36.785505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 801/1628] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:36.802957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2d7b1-ed6c-44f4-a74c-d498bf82fd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc523b011464612b996bad60b7e7e6e, user_id=27abbdf1560341b8b245ca9e7c29e524, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:36.811860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2d7b1-ed6c-44f4-a74c-d498bf82fd2f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:36.934021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 828/1629] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:36 2022] PUT /identity/v3/projects/8fc523b011464612b996bad60b7e7e6e/users/27abbdf1560341b8b245ca9e7c29e524/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:36.935640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb00e16-0939-440f-9549-6e3798b75d6a service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:36.951603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb00e16-0939-440f-9549-6e3798b75d6a service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:36.980420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:37.043368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e384f-fd59-4dd8-a780-373b5b29ad62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:37.044535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e384f-fd59-4dd8-a780-373b5b29ad62 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:37.044964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e384f-fd59-4dd8-a780-373b5b29ad62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:37.045690 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e384f-fd59-4dd8-a780-373b5b29ad62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:37.046747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e384f-fd59-4dd8-a780-373b5b29ad62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:37.049977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e384f-fd59-4dd8-a780-373b5b29ad62 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:37.063913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 802/1630] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:37.067954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2d7b1-ed6c-44f4-a74c-d498bf82fd2f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:37.085418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2d7b1-ed6c-44f4-a74c-d498bf82fd2f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:37.108543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f9259-a3a0-422d-98d8-f4fac3998216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:37.109374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f9259-a3a0-422d-98d8-f4fac3998216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:37.114959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f9259-a3a0-422d-98d8-f4fac3998216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:37.114959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f9259-a3a0-422d-98d8-f4fac3998216 admin admin] PATH_INFO: `/v3/projects/8fc523b011464612b996bad60b7e7e6e/users/27abbdf1560341b8b245ca9e7c29e524/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:37.115198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f9259-a3a0-422d-98d8-f4fac3998216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc523b011464612b996bad60b7e7e6e, user_id=27abbdf1560341b8b245ca9e7c29e524, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:37.120607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f9259-a3a0-422d-98d8-f4fac3998216 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:37.143605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 829/1631] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:37 2022] PUT /identity/v3/projects/8fc523b011464612b996bad60b7e7e6e/users/27abbdf1560341b8b245ca9e7c29e524/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:37.150606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc01d77-9b0f-4b6f-9e4d-36376987b824 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:37.150841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc01d77-9b0f-4b6f-9e4d-36376987b824 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:37.151147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc01d77-9b0f-4b6f-9e4d-36376987b824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:37.182872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9bc01d77-9b0f-4b6f-9e4d-36376987b824 None None] MFA Rules not processed for user `27abbdf1560341b8b245ca9e7c29e524`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:37.184166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9bc01d77-9b0f-4b6f-9e4d-36376987b824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:37.277730 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 803/1632] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:37 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:37.298788 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f9259-a3a0-422d-98d8-f4fac3998216 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:37.395785 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f9259-a3a0-422d-98d8-f4fac3998216 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:37.418908 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f98b4-a940-416f-b752-93da515d4730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:37.419999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f98b4-a940-416f-b752-93da515d4730 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:37.419999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f98b4-a940-416f-b752-93da515d4730 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:37.420239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f98b4-a940-416f-b752-93da515d4730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:37.435609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f98b4-a940-416f-b752-93da515d4730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:37.439353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f98b4-a940-416f-b752-93da515d4730 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:37.464007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 830/1633] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:37 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:38.112695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc01d77-9b0f-4b6f-9e4d-36376987b824 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:38.112695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc01d77-9b0f-4b6f-9e4d-36376987b824 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:38.112695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:38.112695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:38.115271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:38.115271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] PATH_INFO: `/v3/users/b60d28e09b304b1a9a431d58f55373ee` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:38.115271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b60d28e09b304b1a9a431d58f55373ee)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:38.115271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:38.344573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:38.352838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b60d28e09b304b1a9a431d58f55373ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:38.353649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b60d28e09b304b1a9a431d58f55373ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:38.380502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b60d28e09b304b1a9a431d58f55373ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:38.503603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b60d28e09b304b1a9a431d58f55373ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:38.504872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-81f6b4b4-fb4f-404d-acde-ab9e58347e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b60d28e09b304b1a9a431d58f55373ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:38.510247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f98b4-a940-416f-b752-93da515d4730 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:38.529010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f98b4-a940-416f-b752-93da515d4730 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:38.533267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 804/1634] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:37 2022] DELETE /identity/v3/users/b60d28e09b304b1a9a431d58f55373ee => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:38.538077 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e12bbc-5bf1-4264-87cf-6c5fc55f2d23 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:38.538951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e12bbc-5bf1-4264-87cf-6c5fc55f2d23 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:38.539234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e12bbc-5bf1-4264-87cf-6c5fc55f2d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:38.585165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e9e12bbc-5bf1-4264-87cf-6c5fc55f2d23 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:38.592793 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e9e12bbc-5bf1-4264-87cf-6c5fc55f2d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:38.663960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:38.678270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:38.678975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:38.679252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:38.679521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] PATH_INFO: `/v3/users/11f15a53ee034000a29d5ed684c052db` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:38.680815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f15a53ee034000a29d5ed684c052db)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:38.682518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: WARNING py.warnings [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:42:38.682518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: warnings.warn(msg) Oct 04 05:42:38.682518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: Oct 04 05:42:38.683795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:38.738261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 805/1635] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:38.763764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e12bbc-5bf1-4264-87cf-6c5fc55f2d23 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:38.770857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e9e12bbc-5bf1-4264-87cf-6c5fc55f2d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:38.901239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e12bbc-5bf1-4264-87cf-6c5fc55f2d23 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:38.905652 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5166288-0e3e-42c2-93ad-0575c41c5dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:38.906309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5166288-0e3e-42c2-93ad-0575c41c5dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:38.906585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5166288-0e3e-42c2-93ad-0575c41c5dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:38.906854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5166288-0e3e-42c2-93ad-0575c41c5dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:38.917907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5166288-0e3e-42c2-93ad-0575c41c5dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:38.919385 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5166288-0e3e-42c2-93ad-0575c41c5dd4 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:38.935590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 806/1636] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:38.968541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f15a53ee034000a29d5ed684c052db'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:38.992660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f15a53ee034000a29d5ed684c052db'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:38.993330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f15a53ee034000a29d5ed684c052db'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:39.120169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f15a53ee034000a29d5ed684c052db'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:39.120869 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f15a53ee034000a29d5ed684c052db'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:39.141827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 831/1637] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:38 2022] DELETE /identity/v3/users/11f15a53ee034000a29d5ed684c052db => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:42:39.143523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aa334-c13d-41af-844f-7e2213f56832 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:39.143814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aa334-c13d-41af-844f-7e2213f56832 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:39.144092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aa334-c13d-41af-844f-7e2213f56832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:39.172745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-751aa334-c13d-41af-844f-7e2213f56832 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:39.174475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-751aa334-c13d-41af-844f-7e2213f56832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:39.256193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 807/1638] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:39.289705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:39.292866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:39.412562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c78b5-36b7-4451-bf16-81a2ae955ff6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:39.420957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e17f-433a-491c-9c5d-4aa39301bb35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:39.422043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e17f-433a-491c-9c5d-4aa39301bb35 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:39.422489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e17f-433a-491c-9c5d-4aa39301bb35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:39.423101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277e17f-433a-491c-9c5d-4aa39301bb35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:39.450546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277e17f-433a-491c-9c5d-4aa39301bb35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:39.452301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277e17f-433a-491c-9c5d-4aa39301bb35 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:39.463305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 832/1639] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:39.806992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc640-da57-4674-affa-4818921387be None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:39.807544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc640-da57-4674-affa-4818921387be None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:39.807734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc640-da57-4674-affa-4818921387be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:39.820396 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:39.866898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c6bcc640-da57-4674-affa-4818921387be None None] MFA Rules not processed for user `49b61dda6e4a43ca8587e7d12d4ade2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:39.868238 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c6bcc640-da57-4674-affa-4818921387be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:40.046817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e17f-433a-491c-9c5d-4aa39301bb35 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:40.050082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0277e17f-433a-491c-9c5d-4aa39301bb35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:40.062235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 808/1640] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:39 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:40.073018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277e17f-433a-491c-9c5d-4aa39301bb35 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:40.080039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd969f-1cf0-4163-a697-b3f78343bc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:40.081023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd969f-1cf0-4163-a697-b3f78343bc57 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:40.081405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd969f-1cf0-4163-a697-b3f78343bc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:40.082263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd969f-1cf0-4163-a697-b3f78343bc57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:40.083878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd969f-1cf0-4163-a697-b3f78343bc57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:40.087990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd969f-1cf0-4163-a697-b3f78343bc57 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:40.234482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 833/1641] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:40 2022] POST /identity/v3/projects => generated 403 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:40.239131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc640-da57-4674-affa-4818921387be None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:40.263446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc640-da57-4674-affa-4818921387be None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:40.341128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ea3c2-3ee3-4ff5-842f-ea06850aa41d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:40.341128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ea3c2-3ee3-4ff5-842f-ea06850aa41d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:40.341410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ea3c2-3ee3-4ff5-842f-ea06850aa41d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:40.343226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 834/1642] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 05:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:42:40.351073 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b2e56-c97d-4a11-8dc3-f7245f8b44f6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:40.351344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b2e56-c97d-4a11-8dc3-f7245f8b44f6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:40.351500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b2e56-c97d-4a11-8dc3-f7245f8b44f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:40.381301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc500d-0b38-4ec5-8ada-40cf256c4789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:40.381930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc500d-0b38-4ec5-8ada-40cf256c4789 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:40.382236 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc500d-0b38-4ec5-8ada-40cf256c4789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:40.382514 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc500d-0b38-4ec5-8ada-40cf256c4789 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:40.383153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc500d-0b38-4ec5-8ada-40cf256c4789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:40.385103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc500d-0b38-4ec5-8ada-40cf256c4789 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:40.423940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3d7b2e56-c97d-4a11-8dc3-f7245f8b44f6 None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:40.425332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3d7b2e56-c97d-4a11-8dc3-f7245f8b44f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:40.464793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:40.510716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 809/1643] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:40 2022] POST /identity/v3/users => generated 308 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:40.511850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc500d-0b38-4ec5-8ada-40cf256c4789 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:40.514056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fbbc500d-0b38-4ec5-8ada-40cf256c4789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:40.523993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 835/1644] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 05:42:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:42:40.525478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b2e56-c97d-4a11-8dc3-f7245f8b44f6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:40.539876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b2e56-c97d-4a11-8dc3-f7245f8b44f6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:40.542591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc500d-0b38-4ec5-8ada-40cf256c4789 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:40.552234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:40.553877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119b2b9-8eba-4ba7-8c1e-adcfbb416e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:40.554297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:40.554713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:40.555461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119b2b9-8eba-4ba7-8c1e-adcfbb416e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:40.555880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] PATH_INFO: `/v3/projects/1029164654a74b7ea9ae5854836a18d4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:40.556949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119b2b9-8eba-4ba7-8c1e-adcfbb416e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:40.558373 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1029164654a74b7ea9ae5854836a18d4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:40.558599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119b2b9-8eba-4ba7-8c1e-adcfbb416e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:40.559400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119b2b9-8eba-4ba7-8c1e-adcfbb416e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:40.562202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: WARNING py.warnings [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 05:42:40.562202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: warnings.warn(msg) Oct 04 05:42:40.562202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: Oct 04 05:42:40.563763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119b2b9-8eba-4ba7-8c1e-adcfbb416e1c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:40.566110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:40.587767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 836/1645] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:40.587767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66272f70-54c1-43eb-9c0d-7f733c5e0803 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:40.587767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66272f70-54c1-43eb-9c0d-7f733c5e0803 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:40.587767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66272f70-54c1-43eb-9c0d-7f733c5e0803 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:40.591139 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 837/1646] 10.222.0.26 () {54 vars in 912 bytes} [Tue Oct 4 05:42:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:42:40.592090 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66272f70-54c1-43eb-9c0d-7f733c5e0803 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:40.621457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66272f70-54c1-43eb-9c0d-7f733c5e0803 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:40.626858 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f4309-94ac-47ff-bef9-14614df230df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:40.627591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f4309-94ac-47ff-bef9-14614df230df admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:40.627868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f4309-94ac-47ff-bef9-14614df230df admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:40.628140 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f4309-94ac-47ff-bef9-14614df230df admin admin] PATH_INFO: `/v3/projects/b9cd119dd400423299db4ac398dd4ca1/users/f0020830fbb64959ba107dbc4f7f7e3a/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:40.647830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f4309-94ac-47ff-bef9-14614df230df admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9cd119dd400423299db4ac398dd4ca1, user_id=f0020830fbb64959ba107dbc4f7f7e3a, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:40.649613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f4309-94ac-47ff-bef9-14614df230df admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:40.742500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 838/1647] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:40 2022] PUT /identity/v3/projects/b9cd119dd400423299db4ac398dd4ca1/users/f0020830fbb64959ba107dbc4f7f7e3a/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:40.743854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fcb88-17f1-4e5b-bead-9ffaef29ec84 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:40.744149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fcb88-17f1-4e5b-bead-9ffaef29ec84 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:40.744420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fcb88-17f1-4e5b-bead-9ffaef29ec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:40.776731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1029164654a74b7ea9ae5854836a18d4', 'request_id': 'req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:40.797957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6e4fcb88-17f1-4e5b-bead-9ffaef29ec84 None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:40.799240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6e4fcb88-17f1-4e5b-bead-9ffaef29ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:40.815718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1029164654a74b7ea9ae5854836a18d4', 'request_id': 'req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:40.909229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1029164654a74b7ea9ae5854836a18d4', 'request_id': 'req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:40.934054 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 839/1648] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:42:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:40.935225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fcb88-17f1-4e5b-bead-9ffaef29ec84 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:40.936686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] The token cache is being invalidate because project 1029164654a74b7ea9ae5854836a18d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:42:40.937201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:40.937635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6e4fcb88-17f1-4e5b-bead-9ffaef29ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:40.938613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:40.940833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 810/1649] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:40 2022] DELETE /identity/v3/projects/1029164654a74b7ea9ae5854836a18d4 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:40.942078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:40.945797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:41.109295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fcb88-17f1-4e5b-bead-9ffaef29ec84 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:41.111806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf6d92-7da7-4957-adeb-f31d6c9ec2ea admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:41.123391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5c16e-2b95-46ab-b996-7f2dbaca9c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:41.123603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:41.124117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:41.124402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:41.124681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] PATH_INFO: `/v3/projects/845a814bddd847a99ac186b52197aa3f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:41.124944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5c16e-2b95-46ab-b996-7f2dbaca9c50 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:41.126091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5c16e-2b95-46ab-b996-7f2dbaca9c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:41.126494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5c16e-2b95-46ab-b996-7f2dbaca9c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:41.126632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=845a814bddd847a99ac186b52197aa3f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:41.128403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5c16e-2b95-46ab-b996-7f2dbaca9c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:41.128588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:41.135683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5c16e-2b95-46ab-b996-7f2dbaca9c50 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:41.153099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 811/1650] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:41.164045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5c16e-2b95-46ab-b996-7f2dbaca9c50 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:41.169357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3bf5c16e-2b95-46ab-b996-7f2dbaca9c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:41.194460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5c16e-2b95-46ab-b996-7f2dbaca9c50 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:41.211228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:41.212207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:41.212591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:41.212991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] PATH_INFO: `/v3/users/27abbdf1560341b8b245ca9e7c29e524` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:41.217563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=27abbdf1560341b8b245ca9e7c29e524)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:41.219940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:41.302343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '845a814bddd847a99ac186b52197aa3f', 'request_id': 'req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:41.317365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '845a814bddd847a99ac186b52197aa3f', 'request_id': 'req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:41.417432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '845a814bddd847a99ac186b52197aa3f', 'request_id': 'req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:41.434734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] The token cache is being invalidate because project 845a814bddd847a99ac186b52197aa3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:42:41.435010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:41.435683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:41.437236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 840/1651] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:40 2022] DELETE /identity/v3/projects/845a814bddd847a99ac186b52197aa3f => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:41.444287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:41.448682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:41.525749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27abbdf1560341b8b245ca9e7c29e524'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:41.527404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27abbdf1560341b8b245ca9e7c29e524'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:41.544450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc38bb-a3e5-401f-b2f5-728a9c84ce9e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:41.553924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27abbdf1560341b8b245ca9e7c29e524'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:41.554233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad262f-55d3-4774-a4e8-b1f5eac5c156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:41.555057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad262f-55d3-4774-a4e8-b1f5eac5c156 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:41.555584 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad262f-55d3-4774-a4e8-b1f5eac5c156 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:41.556022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad262f-55d3-4774-a4e8-b1f5eac5c156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:41.558510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-adad262f-55d3-4774-a4e8-b1f5eac5c156 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:41.640430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27abbdf1560341b8b245ca9e7c29e524'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:41.641143 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27abbdf1560341b8b245ca9e7c29e524'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:41.661030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 812/1652] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:41 2022] DELETE /identity/v3/users/27abbdf1560341b8b245ca9e7c29e524 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:41.662006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:41.666912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:41.677186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad262f-55d3-4774-a4e8-b1f5eac5c156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:41.681271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad262f-55d3-4774-a4e8-b1f5eac5c156 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:41.711605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 841/1653] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:41.714341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad262f-55d3-4774-a4e8-b1f5eac5c156 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:41.719655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-adad262f-55d3-4774-a4e8-b1f5eac5c156 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:41.787838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70f54b-3c5a-4f4c-95ea-168aaf25a98a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:41.804230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c898282-3d98-4a30-a1fc-36001ce9bb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:41.805067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c898282-3d98-4a30-a1fc-36001ce9bb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:41.805447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c898282-3d98-4a30-a1fc-36001ce9bb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:41.809205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c898282-3d98-4a30-a1fc-36001ce9bb9c admin admin] PATH_INFO: `/v3/projects/b9cd119dd400423299db4ac398dd4ca1/users/f0020830fbb64959ba107dbc4f7f7e3a/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:41.810807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c898282-3d98-4a30-a1fc-36001ce9bb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9cd119dd400423299db4ac398dd4ca1, user_id=f0020830fbb64959ba107dbc4f7f7e3a, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:41.813875 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c898282-3d98-4a30-a1fc-36001ce9bb9c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:41.841143 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 813/1654] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:41 2022] PUT /identity/v3/projects/b9cd119dd400423299db4ac398dd4ca1/users/f0020830fbb64959ba107dbc4f7f7e3a/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:41.880816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad262f-55d3-4774-a4e8-b1f5eac5c156 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:41.986714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:42.049626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:42.050598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:42.051434 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:42.051711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] PATH_INFO: `/v3/users/a0bfec7c8a4b4b3e8d811cac8eb07fae` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:42.055299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0bfec7c8a4b4b3e8d811cac8eb07fae)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:42.057271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:42.224807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:42.325571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0bfec7c8a4b4b3e8d811cac8eb07fae'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:42.383412 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0bfec7c8a4b4b3e8d811cac8eb07fae'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:42.384994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0bfec7c8a4b4b3e8d811cac8eb07fae'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:42.534235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0bfec7c8a4b4b3e8d811cac8eb07fae'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:42.539332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eceeb8f5-3dca-4d33-b3bb-03a700882399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0bfec7c8a4b4b3e8d811cac8eb07fae'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:42.568039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 842/1655] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:41 2022] DELETE /identity/v3/users/a0bfec7c8a4b4b3e8d811cac8eb07fae => generated 0 bytes in 856 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:43.021626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c898282-3d98-4a30-a1fc-36001ce9bb9c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:43.035264 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1c898282-3d98-4a30-a1fc-36001ce9bb9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:43.096288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c898282-3d98-4a30-a1fc-36001ce9bb9c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:43.271782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:43.274703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:43.276801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:43.278749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] PATH_INFO: `/v3/projects/8fc523b011464612b996bad60b7e7e6e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:43.283233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fc523b011464612b996bad60b7e7e6e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:43.285419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:43.483283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc523b011464612b996bad60b7e7e6e', 'request_id': 'req-4ed9e49c-097f-4106-bf98-7e081e51795a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:43.500626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc523b011464612b996bad60b7e7e6e', 'request_id': 'req-4ed9e49c-097f-4106-bf98-7e081e51795a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:43.586661 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:43.616081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc523b011464612b996bad60b7e7e6e', 'request_id': 'req-4ed9e49c-097f-4106-bf98-7e081e51795a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:43.640665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] The token cache is being invalidate because project 8fc523b011464612b996bad60b7e7e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:42:43.648379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:43.648761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4ed9e49c-097f-4106-bf98-7e081e51795a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:43.659382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 814/1656] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:43 2022] DELETE /identity/v3/projects/8fc523b011464612b996bad60b7e7e6e => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:43.662080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50699-9dc3-4362-96b7-c6156d1fe1f8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:43.667482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50699-9dc3-4362-96b7-c6156d1fe1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:43.667979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50699-9dc3-4362-96b7-c6156d1fe1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:43.728660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8ad50699-9dc3-4362-96b7-c6156d1fe1f8 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:43.730166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8ad50699-9dc3-4362-96b7-c6156d1fe1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:43.888150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 843/1657] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:43.905969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1544f-bdac-4384-b1d9-f0fcdc7f0807 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:43.909250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1544f-bdac-4384-b1d9-f0fcdc7f0807 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:43.909604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1544f-bdac-4384-b1d9-f0fcdc7f0807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:43.954231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-aaa1544f-bdac-4384-b1d9-f0fcdc7f0807 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:43.955565 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aaa1544f-bdac-4384-b1d9-f0fcdc7f0807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:43.989205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 815/1658] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:43.993919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad50699-9dc3-4362-96b7-c6156d1fe1f8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:44.007438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad50699-9dc3-4362-96b7-c6156d1fe1f8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:44.022912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c56c5-0905-4c86-8cd5-2f6c2dc52806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:44.023820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c56c5-0905-4c86-8cd5-2f6c2dc52806 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:44.024365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c56c5-0905-4c86-8cd5-2f6c2dc52806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:44.024928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c56c5-0905-4c86-8cd5-2f6c2dc52806 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:44.025776 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c56c5-0905-4c86-8cd5-2f6c2dc52806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:44.026423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c56c5-0905-4c86-8cd5-2f6c2dc52806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:44.027964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:44.031103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c56c5-0905-4c86-8cd5-2f6c2dc52806 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:44.068282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 844/1659] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:44.069439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1544f-bdac-4384-b1d9-f0fcdc7f0807 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:44.085349 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1544f-bdac-4384-b1d9-f0fcdc7f0807 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:44.106916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501dddf-dad7-4e49-90b7-dabda83c6a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:44.107966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501dddf-dad7-4e49-90b7-dabda83c6a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:44.108354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501dddf-dad7-4e49-90b7-dabda83c6a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:44.108741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501dddf-dad7-4e49-90b7-dabda83c6a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:44.109519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501dddf-dad7-4e49-90b7-dabda83c6a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:44.112172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501dddf-dad7-4e49-90b7-dabda83c6a23 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:44.312277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 816/1660] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:44 2022] POST /identity/v3/projects => generated 365 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:42:44.317329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c56c5-0905-4c86-8cd5-2f6c2dc52806 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:44.333138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c56c5-0905-4c86-8cd5-2f6c2dc52806 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:44.495704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015da516-ab18-4e66-92ab-02e03d6b2989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:44.496262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015da516-ab18-4e66-92ab-02e03d6b2989 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:44.496538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015da516-ab18-4e66-92ab-02e03d6b2989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:44.496808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015da516-ab18-4e66-92ab-02e03d6b2989 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:44.497451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015da516-ab18-4e66-92ab-02e03d6b2989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:44.499425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015da516-ab18-4e66-92ab-02e03d6b2989 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:44.604497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045bd42-fb11-4e4b-8e21-361b53fade02 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:44.605363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045bd42-fb11-4e4b-8e21-361b53fade02 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:44.605908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045bd42-fb11-4e4b-8e21-361b53fade02 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:44.608046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 817/1661] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 05:42:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:42:44.621095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636e176-8ebb-4c9d-a3fd-aacbba5d2f1b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:44.621560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636e176-8ebb-4c9d-a3fd-aacbba5d2f1b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:44.621978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636e176-8ebb-4c9d-a3fd-aacbba5d2f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:44.641411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 845/1662] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:44 2022] POST /identity/v3/users => generated 283 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:44.642505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015da516-ab18-4e66-92ab-02e03d6b2989 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:44.657767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015da516-ab18-4e66-92ab-02e03d6b2989 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:44.671452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145cbb8-eb99-4991-a925-f731d2fd0070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:44.672094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145cbb8-eb99-4991-a925-f731d2fd0070 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:44.672338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145cbb8-eb99-4991-a925-f731d2fd0070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:44.678594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4636e176-8ebb-4c9d-a3fd-aacbba5d2f1b None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:44.679866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4636e176-8ebb-4c9d-a3fd-aacbba5d2f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:44.681560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145cbb8-eb99-4991-a925-f731d2fd0070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:44.682648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145cbb8-eb99-4991-a925-f731d2fd0070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:44.685105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145cbb8-eb99-4991-a925-f731d2fd0070 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:44.735900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 846/1663] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:44.740688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145cbb8-eb99-4991-a925-f731d2fd0070 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:44.771708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145cbb8-eb99-4991-a925-f731d2fd0070 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:44.794471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f736f0-b8bc-4f1e-9ff9-87acf80c62a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:44.796337 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f736f0-b8bc-4f1e-9ff9-87acf80c62a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:44.796618 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f736f0-b8bc-4f1e-9ff9-87acf80c62a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:44.799436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f736f0-b8bc-4f1e-9ff9-87acf80c62a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:44.800124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f736f0-b8bc-4f1e-9ff9-87acf80c62a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:44.807379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f736f0-b8bc-4f1e-9ff9-87acf80c62a2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:44.841653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:44.851777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 847/1664] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:44.856779 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f736f0-b8bc-4f1e-9ff9-87acf80c62a2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:44.866388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 818/1665] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 05:42:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:44.874814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5b1ca-1e3e-46ab-8a23-6ff586e10e3d None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:44.875180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5b1ca-1e3e-46ab-8a23-6ff586e10e3d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:44.875653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5b1ca-1e3e-46ab-8a23-6ff586e10e3d None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:44.879605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f736f0-b8bc-4f1e-9ff9-87acf80c62a2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:44.880084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 819/1666] 10.222.0.26 () {54 vars in 912 bytes} [Tue Oct 4 05:42:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:42:44.892471 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266785e-dace-4db9-bfe9-a337fc00f9be None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:44.893105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266785e-dace-4db9-bfe9-a337fc00f9be None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:44.894914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266785e-dace-4db9-bfe9-a337fc00f9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:44.905010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6813b-2da9-4393-aee2-da7128a443a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:44.905442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6813b-2da9-4393-aee2-da7128a443a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:44.906150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6813b-2da9-4393-aee2-da7128a443a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:44.906150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6813b-2da9-4393-aee2-da7128a443a8 admin admin] PATH_INFO: `/v3/projects/36456e80e4ba4067bf06e95e9f06e03e/users/6aa06379b2d74ff6baa34571c12f8d08/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:44.916557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:44.959940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6813b-2da9-4393-aee2-da7128a443a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36456e80e4ba4067bf06e95e9f06e03e, user_id=6aa06379b2d74ff6baa34571c12f8d08, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:44.962681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6813b-2da9-4393-aee2-da7128a443a8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:44.982344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d266785e-dace-4db9-bfe9-a337fc00f9be None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:44.985114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d266785e-dace-4db9-bfe9-a337fc00f9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:45.012647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 820/1667] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:42:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:45.027523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266785e-dace-4db9-bfe9-a337fc00f9be None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:45.029024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d266785e-dace-4db9-bfe9-a337fc00f9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:45.053675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 848/1668] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:44 2022] PUT /identity/v3/projects/36456e80e4ba4067bf06e95e9f06e03e/users/6aa06379b2d74ff6baa34571c12f8d08/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:45.065655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd4278-4791-4e4b-bc69-787c83a1638d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:45.066228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd4278-4791-4e4b-bc69-787c83a1638d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:45.066682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd4278-4791-4e4b-bc69-787c83a1638d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:45.132255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7acd4278-4791-4e4b-bc69-787c83a1638d None None] MFA Rules not processed for user `6aa06379b2d74ff6baa34571c12f8d08`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:45.139324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7acd4278-4791-4e4b-bc69-787c83a1638d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:45.164042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266785e-dace-4db9-bfe9-a337fc00f9be None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:45.235675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 849/1669] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:45.241067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd4278-4791-4e4b-bc69-787c83a1638d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:45.261177 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75608928-09cf-4739-97b3-bb8049a0dced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:45.261798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75608928-09cf-4739-97b3-bb8049a0dced service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:45.262076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75608928-09cf-4739-97b3-bb8049a0dced service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:45.262368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75608928-09cf-4739-97b3-bb8049a0dced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:45.367238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd4278-4791-4e4b-bc69-787c83a1638d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:45.374318 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60ce67-b256-41f9-b510-a6c27a95d975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:45.375037 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60ce67-b256-41f9-b510-a6c27a95d975 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:45.375501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60ce67-b256-41f9-b510-a6c27a95d975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:45.375831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60ce67-b256-41f9-b510-a6c27a95d975 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:45.376550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60ce67-b256-41f9-b510-a6c27a95d975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:45.379979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60ce67-b256-41f9-b510-a6c27a95d975 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:45.429978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75608928-09cf-4739-97b3-bb8049a0dced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:45.431550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75608928-09cf-4739-97b3-bb8049a0dced service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:45.449252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 821/1670] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:45.552056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 850/1671] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:45 2022] POST /identity/v3/projects => generated 365 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:45.556279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75608928-09cf-4739-97b3-bb8049a0dced service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:45.574066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75608928-09cf-4739-97b3-bb8049a0dced service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:45.625845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60ce67-b256-41f9-b510-a6c27a95d975 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:45.627898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5a60ce67-b256-41f9-b510-a6c27a95d975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:45.754691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c846ff8-f7e3-4f4c-950f-9f0665598035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:45.757178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c846ff8-f7e3-4f4c-950f-9f0665598035 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:45.758099 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c846ff8-f7e3-4f4c-950f-9f0665598035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:45.758659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c846ff8-f7e3-4f4c-950f-9f0665598035 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:45.762065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c846ff8-f7e3-4f4c-950f-9f0665598035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:45.767978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c846ff8-f7e3-4f4c-950f-9f0665598035 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:45.830313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60ce67-b256-41f9-b510-a6c27a95d975 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:45.838004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:45.838907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:45.839933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:45.839933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] PATH_INFO: `/v3/projects/e66884ef88e94c9e832ba74f0d41f381` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:45.840958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e66884ef88e94c9e832ba74f0d41f381)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:45.843904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:45.910604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 822/1672] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:45 2022] POST /identity/v3/users => generated 289 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:45.911694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c846ff8-f7e3-4f4c-950f-9f0665598035 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:45.931654 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c846ff8-f7e3-4f4c-950f-9f0665598035 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:45.945674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc73aa3-66bd-4dd6-8e1e-7fb89393bba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:45.946314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc73aa3-66bd-4dd6-8e1e-7fb89393bba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:45.946642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc73aa3-66bd-4dd6-8e1e-7fb89393bba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:45.950840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc73aa3-66bd-4dd6-8e1e-7fb89393bba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:45.954750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc73aa3-66bd-4dd6-8e1e-7fb89393bba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:45.959195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc73aa3-66bd-4dd6-8e1e-7fb89393bba3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:45.992486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 823/1673] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:46.014533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc73aa3-66bd-4dd6-8e1e-7fb89393bba3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:46.046923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e66884ef88e94c9e832ba74f0d41f381', 'request_id': 'req-b615c0c3-b80c-4bf5-a92d-c3e281189923'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:46.048886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc73aa3-66bd-4dd6-8e1e-7fb89393bba3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:46.071966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e66884ef88e94c9e832ba74f0d41f381', 'request_id': 'req-b615c0c3-b80c-4bf5-a92d-c3e281189923'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:46.163635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e66884ef88e94c9e832ba74f0d41f381', 'request_id': 'req-b615c0c3-b80c-4bf5-a92d-c3e281189923'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:46.197811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] The token cache is being invalidate because project e66884ef88e94c9e832ba74f0d41f381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:42:46.198699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:46.200570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b615c0c3-b80c-4bf5-a92d-c3e281189923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:46.211810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 851/1674] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:45 2022] DELETE /identity/v3/projects/e66884ef88e94c9e832ba74f0d41f381 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:46.214559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d633566-5573-4f93-bb17-c53f44702748 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:46.214844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d633566-5573-4f93-bb17-c53f44702748 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:46.215124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d633566-5573-4f93-bb17-c53f44702748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:46.257344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2d633566-5573-4f93-bb17-c53f44702748 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:46.260018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2d633566-5573-4f93-bb17-c53f44702748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:46.267861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf7fbd-feaf-475c-b1e9-6e3095d7ad92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:46.268606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf7fbd-feaf-475c-b1e9-6e3095d7ad92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:46.268889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf7fbd-feaf-475c-b1e9-6e3095d7ad92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:46.269162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf7fbd-feaf-475c-b1e9-6e3095d7ad92 admin admin] PATH_INFO: `/v3/projects/09d60b47b1314bd097984927e060f258/users/67832fa8d2744947ae16c39bb593c016/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:46.280818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 852/1675] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:46.284723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eabb4-2347-43ee-bf70-5ec2f17b8377 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:46.285282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eabb4-2347-43ee-bf70-5ec2f17b8377 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:46.285740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eabb4-2347-43ee-bf70-5ec2f17b8377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:46.303063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf7fbd-feaf-475c-b1e9-6e3095d7ad92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d60b47b1314bd097984927e060f258, user_id=67832fa8d2744947ae16c39bb593c016, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:46.306043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf7fbd-feaf-475c-b1e9-6e3095d7ad92 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:46.334307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7c3eabb4-2347-43ee-bf70-5ec2f17b8377 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:46.338383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7c3eabb4-2347-43ee-bf70-5ec2f17b8377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:46.356226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 853/1676] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:46.361236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3eabb4-2347-43ee-bf70-5ec2f17b8377 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:46.384749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3eabb4-2347-43ee-bf70-5ec2f17b8377 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:46.398907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b040929-4823-439c-959e-a8f6e3019a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:46.400185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b040929-4823-439c-959e-a8f6e3019a74 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:46.401172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b040929-4823-439c-959e-a8f6e3019a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:46.401947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b040929-4823-439c-959e-a8f6e3019a74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:46.403151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b040929-4823-439c-959e-a8f6e3019a74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:46.404342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b040929-4823-439c-959e-a8f6e3019a74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:46.405612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 824/1677] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:45 2022] PUT /identity/v3/projects/09d60b47b1314bd097984927e060f258/users/67832fa8d2744947ae16c39bb593c016/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:46.407596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b040929-4823-439c-959e-a8f6e3019a74 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:46.409344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf7fbd-feaf-475c-b1e9-6e3095d7ad92 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:46.412657 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-98cf7fbd-feaf-475c-b1e9-6e3095d7ad92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:46.453991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 854/1678] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:46.456610 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b040929-4823-439c-959e-a8f6e3019a74 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:46.474549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b040929-4823-439c-959e-a8f6e3019a74 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:46.557568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf7fbd-feaf-475c-b1e9-6e3095d7ad92 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:46.563474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84e38e-1bd9-4889-90cd-282fce8b6823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:46.564324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84e38e-1bd9-4889-90cd-282fce8b6823 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:46.564786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84e38e-1bd9-4889-90cd-282fce8b6823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:46.565194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84e38e-1bd9-4889-90cd-282fce8b6823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:46.566012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84e38e-1bd9-4889-90cd-282fce8b6823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:46.568486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84e38e-1bd9-4889-90cd-282fce8b6823 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:46.595183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 825/1679] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:42:46.598455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84e38e-1bd9-4889-90cd-282fce8b6823 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:46.625319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84e38e-1bd9-4889-90cd-282fce8b6823 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:46.633085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12860a03-9d83-4232-b158-c241b520b1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:46.634052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12860a03-9d83-4232-b158-c241b520b1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:46.634493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12860a03-9d83-4232-b158-c241b520b1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:46.634904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12860a03-9d83-4232-b158-c241b520b1b0 admin admin] PATH_INFO: `/v3/projects/09d60b47b1314bd097984927e060f258/users/67832fa8d2744947ae16c39bb593c016/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:46.637448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12860a03-9d83-4232-b158-c241b520b1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d60b47b1314bd097984927e060f258, user_id=67832fa8d2744947ae16c39bb593c016, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:46.639257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41386f-1d8a-4001-ad82-45e8cdd705e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:46.639896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41386f-1d8a-4001-ad82-45e8cdd705e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:46.640178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41386f-1d8a-4001-ad82-45e8cdd705e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:46.640452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41386f-1d8a-4001-ad82-45e8cdd705e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:46.641948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12860a03-9d83-4232-b158-c241b520b1b0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:46.644125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41386f-1d8a-4001-ad82-45e8cdd705e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:46.655731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41386f-1d8a-4001-ad82-45e8cdd705e6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:46.662442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 826/1680] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:46 2022] PUT /identity/v3/projects/09d60b47b1314bd097984927e060f258/users/67832fa8d2744947ae16c39bb593c016/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:46.673770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780fd8f-811d-489a-9bec-176c9137b462 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:46.674175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780fd8f-811d-489a-9bec-176c9137b462 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:46.674590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780fd8f-811d-489a-9bec-176c9137b462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:46.730969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4780fd8f-811d-489a-9bec-176c9137b462 None None] MFA Rules not processed for user `67832fa8d2744947ae16c39bb593c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:46.736810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4780fd8f-811d-489a-9bec-176c9137b462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:46.786930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 855/1681] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:46 2022] POST /identity/v3/projects => generated 383 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:42:46.792877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41386f-1d8a-4001-ad82-45e8cdd705e6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:46.892180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:46.960784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41386f-1d8a-4001-ad82-45e8cdd705e6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:46.982357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 827/1682] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:46.989616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780fd8f-811d-489a-9bec-176c9137b462 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:46.989835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287a436-e6e3-42ef-aaeb-034f4416fd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:46.991133 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4780fd8f-811d-489a-9bec-176c9137b462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:46.991449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287a436-e6e3-42ef-aaeb-034f4416fd82 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:46.991732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287a436-e6e3-42ef-aaeb-034f4416fd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:46.993017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287a436-e6e3-42ef-aaeb-034f4416fd82 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:46.993017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287a436-e6e3-42ef-aaeb-034f4416fd82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:46.995134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287a436-e6e3-42ef-aaeb-034f4416fd82 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:47.090559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 856/1683] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:46 2022] POST /identity/v3/users => generated 292 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:42:47.096386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287a436-e6e3-42ef-aaeb-034f4416fd82 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:47.124863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287a436-e6e3-42ef-aaeb-034f4416fd82 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:47.131983 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:47.142360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780fd8f-811d-489a-9bec-176c9137b462 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:47.145290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa982b-a48e-496e-a9b5-a98e48a8f81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:47.146824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa982b-a48e-496e-a9b5-a98e48a8f81b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:47.152060 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa982b-a48e-496e-a9b5-a98e48a8f81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:47.152060 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa982b-a48e-496e-a9b5-a98e48a8f81b admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:47.152691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa982b-a48e-496e-a9b5-a98e48a8f81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:47.154808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa982b-a48e-496e-a9b5-a98e48a8f81b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:47.196437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 857/1684] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:47.201276 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa982b-a48e-496e-a9b5-a98e48a8f81b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:47.227306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa982b-a48e-496e-a9b5-a98e48a8f81b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:47.244166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd6107-e7b0-41b3-9e24-45d51891c9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:47.244791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd6107-e7b0-41b3-9e24-45d51891c9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:47.246811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd6107-e7b0-41b3-9e24-45d51891c9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:47.247085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd6107-e7b0-41b3-9e24-45d51891c9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:47.249057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd6107-e7b0-41b3-9e24-45d51891c9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:47.257393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd6107-e7b0-41b3-9e24-45d51891c9e0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:47.286241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 858/1685] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:47.289317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd6107-e7b0-41b3-9e24-45d51891c9e0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:47.299247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9130e-a6a2-45ab-8f60-4de37cf75bea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:47.300427 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9130e-a6a2-45ab-8f60-4de37cf75bea service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:47.301273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9130e-a6a2-45ab-8f60-4de37cf75bea service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:47.301549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9130e-a6a2-45ab-8f60-4de37cf75bea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:47.351494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd6107-e7b0-41b3-9e24-45d51891c9e0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:47.364080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b17b7-abc9-4d53-8d64-a34493c09adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:47.365557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b17b7-abc9-4d53-8d64-a34493c09adf admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:47.365990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b17b7-abc9-4d53-8d64-a34493c09adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:47.366827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b17b7-abc9-4d53-8d64-a34493c09adf admin admin] PATH_INFO: `/v3/projects/b47daea4aa234222a65a1e0c934fe745/users/697cbc2fb7724107a02a017a692b6e4a/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:47.386935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b17b7-abc9-4d53-8d64-a34493c09adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47daea4aa234222a65a1e0c934fe745, user_id=697cbc2fb7724107a02a017a692b6e4a, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:47.391268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b17b7-abc9-4d53-8d64-a34493c09adf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:47.457809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9130e-a6a2-45ab-8f60-4de37cf75bea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:47.460657 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9130e-a6a2-45ab-8f60-4de37cf75bea service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:47.491865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 828/1686] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:42:47.491865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86586ec-77f2-4902-975a-9a545da6174a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:47.491865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86586ec-77f2-4902-975a-9a545da6174a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:47.491865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86586ec-77f2-4902-975a-9a545da6174a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:47.526212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 859/1687] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:47 2022] PUT /identity/v3/projects/b47daea4aa234222a65a1e0c934fe745/users/697cbc2fb7724107a02a017a692b6e4a/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:47.532672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d86586ec-77f2-4902-975a-9a545da6174a None None] MFA Rules not processed for user `f0020830fbb64959ba107dbc4f7f7e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:47.534007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d86586ec-77f2-4902-975a-9a545da6174a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:47.540978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ace5e5-32e2-4a1c-bb51-d3ff0119d639 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:47.541830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ace5e5-32e2-4a1c-bb51-d3ff0119d639 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:47.542312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ace5e5-32e2-4a1c-bb51-d3ff0119d639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:47.627749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:47.640802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a4ace5e5-32e2-4a1c-bb51-d3ff0119d639 None None] MFA Rules not processed for user `697cbc2fb7724107a02a017a692b6e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:47.645794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a4ace5e5-32e2-4a1c-bb51-d3ff0119d639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:47.683514 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 829/1688] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:47 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:47.820754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 860/1689] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:42:47.827539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86586ec-77f2-4902-975a-9a545da6174a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:47.962537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86586ec-77f2-4902-975a-9a545da6174a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:47.979949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f783fb-ae62-4ad5-8b8c-6057b1c44d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:47.979949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f783fb-ae62-4ad5-8b8c-6057b1c44d20 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:47.979949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f783fb-ae62-4ad5-8b8c-6057b1c44d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:47.979949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f783fb-ae62-4ad5-8b8c-6057b1c44d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:47.980335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f783fb-ae62-4ad5-8b8c-6057b1c44d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:47.981744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f783fb-ae62-4ad5-8b8c-6057b1c44d20 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:48.152431 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 830/1690] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:47 2022] POST /identity/v3/projects => generated 381 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:48.156486 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ace5e5-32e2-4a1c-bb51-d3ff0119d639 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:48.172405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ace5e5-32e2-4a1c-bb51-d3ff0119d639 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:48.319997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d696a-eeb7-42d0-9c74-1704db4100d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:48.321089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d696a-eeb7-42d0-9c74-1704db4100d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:48.323993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d696a-eeb7-42d0-9c74-1704db4100d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:48.324656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d696a-eeb7-42d0-9c74-1704db4100d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:48.329203 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d696a-eeb7-42d0-9c74-1704db4100d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:48.333067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d696a-eeb7-42d0-9c74-1704db4100d9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:48.501989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:48.509324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 861/1691] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:48 2022] POST /identity/v3/users => generated 297 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:42:48.515750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f783fb-ae62-4ad5-8b8c-6057b1c44d20 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:48.540792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f783fb-ae62-4ad5-8b8c-6057b1c44d20 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:48.547654 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb3962-fdac-4dc9-9159-49e698b52774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:48.548866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3962-fdac-4dc9-9159-49e698b52774 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:48.549357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3962-fdac-4dc9-9159-49e698b52774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:48.549829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3962-fdac-4dc9-9159-49e698b52774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:48.552093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb3962-fdac-4dc9-9159-49e698b52774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:48.556169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb3962-fdac-4dc9-9159-49e698b52774 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:48.586866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 831/1692] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:48.588137 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d696a-eeb7-42d0-9c74-1704db4100d9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:48.616029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d696a-eeb7-42d0-9c74-1704db4100d9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:48.634460 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b62bf-610a-49de-a888-5e7b9d9efd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:48.636511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b62bf-610a-49de-a888-5e7b9d9efd88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:48.639353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b62bf-610a-49de-a888-5e7b9d9efd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:48.640813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b62bf-610a-49de-a888-5e7b9d9efd88 admin admin] PATH_INFO: `/v3/projects/d76cdc535cb848d5afdff253e48c2b7e/users/3334c5fa7b07462a9199df67c5f179ee/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:48.701094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b62bf-610a-49de-a888-5e7b9d9efd88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d76cdc535cb848d5afdff253e48c2b7e, user_id=3334c5fa7b07462a9199df67c5f179ee, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:48.711899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b62bf-610a-49de-a888-5e7b9d9efd88 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:48.803185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 862/1693] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:48 2022] PUT /identity/v3/projects/d76cdc535cb848d5afdff253e48c2b7e/users/3334c5fa7b07462a9199df67c5f179ee/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:48.808850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb3962-fdac-4dc9-9159-49e698b52774 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:48.834383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb3962-fdac-4dc9-9159-49e698b52774 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:48.861142 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:48.971351 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86011ce5-251f-448c-9f02-daa2fe0a8c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:48.972365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86011ce5-251f-448c-9f02-daa2fe0a8c49 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:48.972895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86011ce5-251f-448c-9f02-daa2fe0a8c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:48.973363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86011ce5-251f-448c-9f02-daa2fe0a8c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:48.974237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86011ce5-251f-448c-9f02-daa2fe0a8c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:48.976021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86011ce5-251f-448c-9f02-daa2fe0a8c49 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:49.004023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 832/1694] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:49.012078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8ce4b-a955-4b1e-b9d7-eeb75c4f9d2a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:49.012078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8ce4b-a955-4b1e-b9d7-eeb75c4f9d2a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:49.012877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8ce4b-a955-4b1e-b9d7-eeb75c4f9d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:49.013475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86011ce5-251f-448c-9f02-daa2fe0a8c49 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:49.042588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86011ce5-251f-448c-9f02-daa2fe0a8c49 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:49.058195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193da7d-8f94-4a3b-9688-b47f48848a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:49.059034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193da7d-8f94-4a3b-9688-b47f48848a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:49.059552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193da7d-8f94-4a3b-9688-b47f48848a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:49.059961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193da7d-8f94-4a3b-9688-b47f48848a80 admin admin] PATH_INFO: `/v3/projects/d76cdc535cb848d5afdff253e48c2b7e/users/3334c5fa7b07462a9199df67c5f179ee/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:49.067654 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193da7d-8f94-4a3b-9688-b47f48848a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d76cdc535cb848d5afdff253e48c2b7e, user_id=3334c5fa7b07462a9199df67c5f179ee, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:49.070251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fcc8ce4b-a955-4b1e-b9d7-eeb75c4f9d2a None None] MFA Rules not processed for user `c66c7ddbf18d4f1fb8c890f548031f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:49.070726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193da7d-8f94-4a3b-9688-b47f48848a80 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:49.073927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fcc8ce4b-a955-4b1e-b9d7-eeb75c4f9d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:49.101760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 833/1695] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:49 2022] PUT /identity/v3/projects/d76cdc535cb848d5afdff253e48c2b7e/users/3334c5fa7b07462a9199df67c5f179ee/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:49.114118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc006842-b60a-4ce6-8e7a-42a4e069d1ff None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:49.114515 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc006842-b60a-4ce6-8e7a-42a4e069d1ff None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:49.119029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc006842-b60a-4ce6-8e7a-42a4e069d1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:49.223203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dc006842-b60a-4ce6-8e7a-42a4e069d1ff None None] MFA Rules not processed for user `3334c5fa7b07462a9199df67c5f179ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:49.228333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dc006842-b60a-4ce6-8e7a-42a4e069d1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:49.261216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 863/1696] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:49.262684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8ce4b-a955-4b1e-b9d7-eeb75c4f9d2a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:49.302390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8ce4b-a955-4b1e-b9d7-eeb75c4f9d2a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:49.377621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:49.418862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 834/1697] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:42:49.420469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9eb2-320f-4d8e-8383-97a6eebc697a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:49.421045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9eb2-320f-4d8e-8383-97a6eebc697a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:49.421510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9eb2-320f-4d8e-8383-97a6eebc697a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:49.454596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:49.455818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:49.455818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:49.455818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] PATH_INFO: `/v3/users/6aa06379b2d74ff6baa34571c12f8d08` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:49.460198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa06379b2d74ff6baa34571c12f8d08)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:49.464642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:49.505085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e88a9eb2-320f-4d8e-8383-97a6eebc697a None None] MFA Rules not processed for user `6f1635c9c6cf4fb092c0f03a02e36174`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:49.506710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e88a9eb2-320f-4d8e-8383-97a6eebc697a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:49.723748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa06379b2d74ff6baa34571c12f8d08'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:49.727537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 835/1698] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:49.728554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a9eb2-320f-4d8e-8383-97a6eebc697a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:49.736054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e88a9eb2-320f-4d8e-8383-97a6eebc697a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:49.778853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa06379b2d74ff6baa34571c12f8d08'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:49.782429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa06379b2d74ff6baa34571c12f8d08'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:49.869071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa06379b2d74ff6baa34571c12f8d08'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:49.869995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa06379b2d74ff6baa34571c12f8d08'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:49.895015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 864/1699] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:49 2022] DELETE /identity/v3/users/6aa06379b2d74ff6baa34571c12f8d08 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:49.897261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:49.903934 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:49.945348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a9eb2-320f-4d8e-8383-97a6eebc697a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:50.065164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f59d2-47f4-4c8c-b264-001fb915f1f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:50.066174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f59d2-47f4-4c8c-b264-001fb915f1f0 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:50.066600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f59d2-47f4-4c8c-b264-001fb915f1f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:50.067015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f59d2-47f4-4c8c-b264-001fb915f1f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:50.069294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-263f59d2-47f4-4c8c-b264-001fb915f1f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:50.104777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73949f2b-bf49-4c16-bd2d-39e7d31bf89b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:50.117176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788f77b-9897-4f30-be8d-46ccadce658a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:50.117806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788f77b-9897-4f30-be8d-46ccadce658a service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:50.118085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788f77b-9897-4f30-be8d-46ccadce658a service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:50.118379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788f77b-9897-4f30-be8d-46ccadce658a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:50.120331 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6788f77b-9897-4f30-be8d-46ccadce658a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:50.149059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:50.164966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f59d2-47f4-4c8c-b264-001fb915f1f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:50.166633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f59d2-47f4-4c8c-b264-001fb915f1f0 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:50.194610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 836/1700] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:50.196311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f11f6-86a4-4d2e-921b-9a4fdecffdd0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:50.196606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f11f6-86a4-4d2e-921b-9a4fdecffdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:50.196886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f11f6-86a4-4d2e-921b-9a4fdecffdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:50.245005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ce1f11f6-86a4-4d2e-921b-9a4fdecffdd0 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:50.249153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:50.252032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ce1f11f6-86a4-4d2e-921b-9a4fdecffdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:50.281047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788f77b-9897-4f30-be8d-46ccadce658a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:50.291784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788f77b-9897-4f30-be8d-46ccadce658a service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:50.326846 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 865/1701] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:50.327950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788f77b-9897-4f30-be8d-46ccadce658a service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:50.337411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6788f77b-9897-4f30-be8d-46ccadce658a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:50.383816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788f77b-9897-4f30-be8d-46ccadce658a service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:50.408466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ddbfe-47f1-4c8b-8d1c-69ebe22ebbec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:50.409761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ddbfe-47f1-4c8b-8d1c-69ebe22ebbec service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:50.410262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ddbfe-47f1-4c8b-8d1c-69ebe22ebbec service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:50.410639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ddbfe-47f1-4c8b-8d1c-69ebe22ebbec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:50.412742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d47ddbfe-47f1-4c8b-8d1c-69ebe22ebbec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:50.491364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 837/1702] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:50.543722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f11f6-86a4-4d2e-921b-9a4fdecffdd0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:50.556456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f11f6-86a4-4d2e-921b-9a4fdecffdd0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:50.571218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e52a18-cc73-4b8a-a290-cdf9ca9d9d0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:50.571673 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e52a18-cc73-4b8a-a290-cdf9ca9d9d0b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:50.571958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e52a18-cc73-4b8a-a290-cdf9ca9d9d0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:50.572231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e52a18-cc73-4b8a-a290-cdf9ca9d9d0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:50.574160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-01e52a18-cc73-4b8a-a290-cdf9ca9d9d0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:50.610618 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ddbfe-47f1-4c8b-8d1c-69ebe22ebbec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:50.612399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ddbfe-47f1-4c8b-8d1c-69ebe22ebbec service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:50.640749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 866/1703] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:50.723823 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e52a18-cc73-4b8a-a290-cdf9ca9d9d0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:50.725740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e52a18-cc73-4b8a-a290-cdf9ca9d9d0b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:50.729857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:50.747166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 838/1704] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:42:50.955593 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ddbfe-47f1-4c8b-8d1c-69ebe22ebbec service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:50.958324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d47ddbfe-47f1-4c8b-8d1c-69ebe22ebbec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:50.989112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612e8b2-6b7e-4c6c-b9a5-84e0a207d0ef None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:50.994546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612e8b2-6b7e-4c6c-b9a5-84e0a207d0ef None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:50.994546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612e8b2-6b7e-4c6c-b9a5-84e0a207d0ef None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:50.994546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 839/1705] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:42:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:42:50.997335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:51.091519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ed79b-26bf-4be6-a219-c015700bddcc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:51.092237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ed79b-26bf-4be6-a219-c015700bddcc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:51.093134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ed79b-26bf-4be6-a219-c015700bddcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:51.139211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ddbfe-47f1-4c8b-8d1c-69ebe22ebbec service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:51.158117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb0def-db0c-4d09-8284-9e7d7091e7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:51.158675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb0def-db0c-4d09-8284-9e7d7091e7ea service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:51.158960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb0def-db0c-4d09-8284-9e7d7091e7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:51.159235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb0def-db0c-4d09-8284-9e7d7091e7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:51.167423 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0a8ed79b-26bf-4be6-a219-c015700bddcc None None] MFA Rules not processed for user `e95901cce5d448acb2571976163868a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:51.168854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0a8ed79b-26bf-4be6-a219-c015700bddcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:51.196424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb0def-db0c-4d09-8284-9e7d7091e7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:51.197975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb0def-db0c-4d09-8284-9e7d7091e7ea service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:51.226686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 867/1706] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:51.401129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 840/1707] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:42:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:42:51.475301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb0def-db0c-4d09-8284-9e7d7091e7ea service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:51.476896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0feb0def-db0c-4d09-8284-9e7d7091e7ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:51.505297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb0def-db0c-4d09-8284-9e7d7091e7ea service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:51.512267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaac0d-9cc7-471e-b440-f301c00e02fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:51.514366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaac0d-9cc7-471e-b440-f301c00e02fe service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:51.515187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaac0d-9cc7-471e-b440-f301c00e02fe service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:51.515187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaac0d-9cc7-471e-b440-f301c00e02fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:51.527411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaac0d-9cc7-471e-b440-f301c00e02fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:51.529284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaac0d-9cc7-471e-b440-f301c00e02fe service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:51.559516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 868/1708] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:52.028848 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:53.689239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ed79b-26bf-4be6-a219-c015700bddcc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:53.690843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0a8ed79b-26bf-4be6-a219-c015700bddcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:53.719790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ed79b-26bf-4be6-a219-c015700bddcc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:53.733064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:53.733764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:53.734046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:53.734348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] PATH_INFO: `/v3/projects/36456e80e4ba4067bf06e95e9f06e03e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:53.735678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36456e80e4ba4067bf06e95e9f06e03e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:53.738481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:53.931829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36456e80e4ba4067bf06e95e9f06e03e', 'request_id': 'req-2ca5b684-9580-4fb9-8e39-8d290cb77f09'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:53.959166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36456e80e4ba4067bf06e95e9f06e03e', 'request_id': 'req-2ca5b684-9580-4fb9-8e39-8d290cb77f09'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.091523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36456e80e4ba4067bf06e95e9f06e03e', 'request_id': 'req-2ca5b684-9580-4fb9-8e39-8d290cb77f09'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.103952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] The token cache is being invalidate because project 36456e80e4ba4067bf06e95e9f06e03e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:42:54.104235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.105569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2ca5b684-9580-4fb9-8e39-8d290cb77f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.107355 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 841/1709] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:53 2022] DELETE /identity/v3/projects/36456e80e4ba4067bf06e95e9f06e03e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:54.111417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaac0d-9cc7-471e-b440-f301c00e02fe service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:54.125803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaac0d-9cc7-471e-b440-f301c00e02fe service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:54.137609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb0724-e61d-4752-9e54-2c9696949a25 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:54.138417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb0724-e61d-4752-9e54-2c9696949a25 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:54.138882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb0724-e61d-4752-9e54-2c9696949a25 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:54.141285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 842/1710] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:42:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:42:54.163089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdbb6-69d6-4203-a560-c94eab070099 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:54.165247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdbb6-69d6-4203-a560-c94eab070099 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:54.166904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdbb6-69d6-4203-a560-c94eab070099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:54.237616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-299fdbb6-69d6-4203-a560-c94eab070099 None None] MFA Rules not processed for user `e95901cce5d448acb2571976163868a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:54.238910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-299fdbb6-69d6-4203-a560-c94eab070099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:54.277860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:54.279035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:54.279035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:54.279306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] PATH_INFO: `/v3/users/67832fa8d2744947ae16c39bb593c016` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:54.281317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67832fa8d2744947ae16c39bb593c016)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:54.283750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:54.397145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 843/1711] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:42:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:42:54.399576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd657d2-d34f-4fb2-8218-9894c1ebbdd4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:54.401049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd657d2-d34f-4fb2-8218-9894c1ebbdd4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:54.402384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd657d2-d34f-4fb2-8218-9894c1ebbdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:54.481483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:54.484561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4bd657d2-d34f-4fb2-8218-9894c1ebbdd4 None None] MFA Rules not processed for user `9d1d5de04c894a2ab780bcdee15f5c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:54.486284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4bd657d2-d34f-4fb2-8218-9894c1ebbdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:54.583143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67832fa8d2744947ae16c39bb593c016'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.586097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:54.588404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 844/1712] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:54.590235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd657d2-d34f-4fb2-8218-9894c1ebbdd4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:54.598244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4bd657d2-d34f-4fb2-8218-9894c1ebbdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:54.616238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67832fa8d2744947ae16c39bb593c016'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.616977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67832fa8d2744947ae16c39bb593c016'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.641105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd657d2-d34f-4fb2-8218-9894c1ebbdd4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:54.648368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:54.649280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:54.649714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:54.650421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] PATH_INFO: `/v3/users/49b61dda6e4a43ca8587e7d12d4ade2e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:54.652972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b61dda6e4a43ca8587e7d12d4ade2e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:54.655416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:54.740684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67832fa8d2744947ae16c39bb593c016'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.741902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67832fa8d2744947ae16c39bb593c016'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.766806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 869/1713] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:54 2022] DELETE /identity/v3/users/67832fa8d2744947ae16c39bb593c016 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:54.767733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:54.772770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:54.924767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b61dda6e4a43ca8587e7d12d4ade2e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.926504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b61dda6e4a43ca8587e7d12d4ade2e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.962701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b61dda6e4a43ca8587e7d12d4ade2e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:54.967818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113aaaf8-4db1-4efc-901a-6730cb064be3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:54.974917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84053c1-7754-443d-bde1-f74a6bbe3be8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:54.976513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84053c1-7754-443d-bde1-f74a6bbe3be8 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:54.976945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84053c1-7754-443d-bde1-f74a6bbe3be8 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:54.977314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84053c1-7754-443d-bde1-f74a6bbe3be8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:54.980133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e84053c1-7754-443d-bde1-f74a6bbe3be8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:55.000927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84053c1-7754-443d-bde1-f74a6bbe3be8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:55.016720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84053c1-7754-443d-bde1-f74a6bbe3be8 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:55.037188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b61dda6e4a43ca8587e7d12d4ade2e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.037967 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b61dda6e4a43ca8587e7d12d4ade2e'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.064856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 870/1714] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:55.066704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 845/1715] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:54 2022] DELETE /identity/v3/users/49b61dda6e4a43ca8587e7d12d4ade2e => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:55.066840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf14847-de35-4276-8448-76eed93c7200 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:55.067017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf14847-de35-4276-8448-76eed93c7200 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:55.067287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf14847-de35-4276-8448-76eed93c7200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:55.070544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:55.072137 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:55.103790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7bf14847-de35-4276-8448-76eed93c7200 None None] MFA Rules not processed for user `9d1d5de04c894a2ab780bcdee15f5c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:55.106256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7bf14847-de35-4276-8448-76eed93c7200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:55.200659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1e531-46bb-4433-a5f0-5383d0de712b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:55.207592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:55.209827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:55.210262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:55.210638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] PATH_INFO: `/v3/users/697cbc2fb7724107a02a017a692b6e4a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:55.213052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] RBAC: Authorizing `identity:delete_user(user_id=697cbc2fb7724107a02a017a692b6e4a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:55.216228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:55.216852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 871/1716] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:55.217833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf14847-de35-4276-8448-76eed93c7200 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:55.221458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7bf14847-de35-4276-8448-76eed93c7200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:55.263346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf14847-de35-4276-8448-76eed93c7200 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:55.275605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:55.276597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:55.276597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:55.276942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] PATH_INFO: `/v3/users/cd8486004f1141218103f1add322fca9` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:55.281343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd8486004f1141218103f1add322fca9)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:55.283325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:55.530057 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '697cbc2fb7724107a02a017a692b6e4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.530445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd8486004f1141218103f1add322fca9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.532060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '697cbc2fb7724107a02a017a692b6e4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.594687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '697cbc2fb7724107a02a017a692b6e4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.602746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd8486004f1141218103f1add322fca9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.608813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd8486004f1141218103f1add322fca9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.718103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '697cbc2fb7724107a02a017a692b6e4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.719205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '697cbc2fb7724107a02a017a692b6e4a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.754018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 846/1717] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:55 2022] DELETE /identity/v3/users/697cbc2fb7724107a02a017a692b6e4a => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:55.755162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:55.756783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:55.854771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd8486004f1141218103f1add322fca9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.858434 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4faf3a05-3717-42b8-8a73-9c1130b46304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd8486004f1141218103f1add322fca9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:55.881882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 872/1718] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:55 2022] DELETE /identity/v3/users/cd8486004f1141218103f1add322fca9 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:55.895452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a135ad-79fd-4b78-978f-543722627392 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:55.895733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a135ad-79fd-4b78-978f-543722627392 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:55.901174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a135ad-79fd-4b78-978f-543722627392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:55.944863 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4c6bd-4189-43d7-bcf3-89d7dbef029a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:55.966037 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f7a135ad-79fd-4b78-978f-543722627392 None None] MFA Rules not processed for user `537981b1b3a14cb4b10f336ef0362358`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:55.967363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f7a135ad-79fd-4b78-978f-543722627392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:55.980662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:56.081975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8373aad-769f-4a7f-bc03-b1bd46578af8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:56.082872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8373aad-769f-4a7f-bc03-b1bd46578af8 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:56.083318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8373aad-769f-4a7f-bc03-b1bd46578af8 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:56.083884 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8373aad-769f-4a7f-bc03-b1bd46578af8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:56.086000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b8373aad-769f-4a7f-bc03-b1bd46578af8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.102686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 873/1719] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:56.108013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a135ad-79fd-4b78-978f-543722627392 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:56.109763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f7a135ad-79fd-4b78-978f-543722627392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.178943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8373aad-769f-4a7f-bc03-b1bd46578af8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:56.180822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8373aad-769f-4a7f-bc03-b1bd46578af8 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:56.208299 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 847/1720] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:56.210200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cb45d-531f-47b1-8535-d6210f0039dd None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:56.210512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cb45d-531f-47b1-8535-d6210f0039dd None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:56.210758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cb45d-531f-47b1-8535-d6210f0039dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:56.233016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a135ad-79fd-4b78-978f-543722627392 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:56.234083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c88cb45d-531f-47b1-8535-d6210f0039dd None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:56.235803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c88cb45d-531f-47b1-8535-d6210f0039dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.240627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:56.242063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:56.242886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:56.243388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] PATH_INFO: `/v3/projects/eef5caf8a6d24a65a5758762c5bdfc3a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:56.245347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 848/1721] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:56.246902 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cb45d-531f-47b1-8535-d6210f0039dd None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:56.254795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef5caf8a6d24a65a5758762c5bdfc3a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:56.256775 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:56.271574 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:56.272490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c88cb45d-531f-47b1-8535-d6210f0039dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.295121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cb45d-531f-47b1-8535-d6210f0039dd None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:56.305798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:56.306971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:56.307461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:56.307925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] PATH_INFO: `/v3/projects/09d60b47b1314bd097984927e060f258` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:56.309525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d60b47b1314bd097984927e060f258)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:56.312264 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:56.402158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef5caf8a6d24a65a5758762c5bdfc3a', 'request_id': 'req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:56.410763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef5caf8a6d24a65a5758762c5bdfc3a', 'request_id': 'req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:56.489892 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d60b47b1314bd097984927e060f258', 'request_id': 'req-265d4f5c-a933-4f18-960e-4bd44405ab90'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:56.494935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef5caf8a6d24a65a5758762c5bdfc3a', 'request_id': 'req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:56.510290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] The token cache is being invalidate because project eef5caf8a6d24a65a5758762c5bdfc3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:42:56.510734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:56.512104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b12bbb49-82b2-4ca5-ac71-c91006f6ba2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:56.517143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 874/1722] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:56 2022] DELETE /identity/v3/projects/eef5caf8a6d24a65a5758762c5bdfc3a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:56.517143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bac7b-0511-469d-bf51-34e2b75dbc8c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:56.517143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bac7b-0511-469d-bf51-34e2b75dbc8c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:56.517143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bac7b-0511-469d-bf51-34e2b75dbc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:56.527670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d60b47b1314bd097984927e060f258', 'request_id': 'req-265d4f5c-a933-4f18-960e-4bd44405ab90'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:56.538754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-263bac7b-0511-469d-bf51-34e2b75dbc8c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:56.543534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-263bac7b-0511-469d-bf51-34e2b75dbc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.608108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d60b47b1314bd097984927e060f258', 'request_id': 'req-265d4f5c-a933-4f18-960e-4bd44405ab90'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:56.633436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 875/1723] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:56.634837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] The token cache is being invalidate because project 09d60b47b1314bd097984927e060f258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:42:56.635872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fb5c5-3e1f-4682-b621-3f193a95584d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:56.636113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:56.636282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fb5c5-3e1f-4682-b621-3f193a95584d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:56.636282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fb5c5-3e1f-4682-b621-3f193a95584d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:56.636558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:56.641357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 849/1724] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:56 2022] DELETE /identity/v3/projects/09d60b47b1314bd097984927e060f258 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:56.641357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:56.641357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.667836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bd2fb5c5-3e1f-4682-b621-3f193a95584d None None] MFA Rules not processed for user `c66c7ddbf18d4f1fb8c890f548031f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:56.669668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bd2fb5c5-3e1f-4682-b621-3f193a95584d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.762982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 876/1725] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:56.763930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fb5c5-3e1f-4682-b621-3f193a95584d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:56.768627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bd2fb5c5-3e1f-4682-b621-3f193a95584d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.840097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fb5c5-3e1f-4682-b621-3f193a95584d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:56.841876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d4f5c-a933-4f18-960e-4bd44405ab90 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:56.846673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f1a78-e1da-426b-ba32-e441269d4583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:56.848137 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f1a78-e1da-426b-ba32-e441269d4583 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:56.848491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f1a78-e1da-426b-ba32-e441269d4583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:56.848829 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f1a78-e1da-426b-ba32-e441269d4583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:56.852332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-232f1a78-e1da-426b-ba32-e441269d4583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.853881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c90c3-a6f2-4457-b67a-b359d70fe1db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:56.854716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c90c3-a6f2-4457-b67a-b359d70fe1db service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:56.855143 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c90c3-a6f2-4457-b67a-b359d70fe1db service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:56.855557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c90c3-a6f2-4457-b67a-b359d70fe1db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:56.858466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-674c90c3-a6f2-4457-b67a-b359d70fe1db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.875632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f1a78-e1da-426b-ba32-e441269d4583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:56.879849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f1a78-e1da-426b-ba32-e441269d4583 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:56.908464 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 877/1726] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:56.912813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5d5b6-0fa7-466a-bc98-30a0dd48be2c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:56.913187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5d5b6-0fa7-466a-bc98-30a0dd48be2c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:56.913536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5d5b6-0fa7-466a-bc98-30a0dd48be2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:56.953800 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c90c3-a6f2-4457-b67a-b359d70fe1db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:56.957631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c90c3-a6f2-4457-b67a-b359d70fe1db service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:56.959254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-50e5d5b6-0fa7-466a-bc98-30a0dd48be2c None None] MFA Rules not processed for user `6f1635c9c6cf4fb092c0f03a02e36174`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:56.962007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-50e5d5b6-0fa7-466a-bc98-30a0dd48be2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:56.972348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 850/1727] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:56.988810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c90c3-a6f2-4457-b67a-b359d70fe1db service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:56.994096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-674c90c3-a6f2-4457-b67a-b359d70fe1db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:57.027613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c90c3-a6f2-4457-b67a-b359d70fe1db service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:57.040849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:57.041492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.041779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.042050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] PATH_INFO: `/v3/users/f0020830fbb64959ba107dbc4f7f7e3a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.045731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0020830fbb64959ba107dbc4f7f7e3a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:57.058744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:57.096006 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 878/1728] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:57.097847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19f536-3737-4b29-837c-6a5d440acb34 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.098253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19f536-3737-4b29-837c-6a5d440acb34 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.098646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19f536-3737-4b29-837c-6a5d440acb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.123238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9f19f536-3737-4b29-837c-6a5d440acb34 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:57.124798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9f19f536-3737-4b29-837c-6a5d440acb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:57.139046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 879/1729] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:57.142710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19f536-3737-4b29-837c-6a5d440acb34 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:57.142710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9f19f536-3737-4b29-837c-6a5d440acb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:57.158604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19f536-3737-4b29-837c-6a5d440acb34 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:57.168624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383c806-f959-486c-823c-564050e6fa4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:57.169350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383c806-f959-486c-823c-564050e6fa4f service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.170201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383c806-f959-486c-823c-564050e6fa4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.170538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383c806-f959-486c-823c-564050e6fa4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.172845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8383c806-f959-486c-823c-564050e6fa4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:57.190349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383c806-f959-486c-823c-564050e6fa4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:57.192511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383c806-f959-486c-823c-564050e6fa4f service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:57.205892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 880/1730] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:57.208282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c8fd9-2365-403c-a39f-ef1c6d49d8cd None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.208574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c8fd9-2365-403c-a39f-ef1c6d49d8cd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.208837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c8fd9-2365-403c-a39f-ef1c6d49d8cd None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.210444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 881/1731] 10.222.0.26 () {54 vars in 912 bytes} [Tue Oct 4 05:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:42:57.211614 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3811f56-5d4f-4df4-9dab-59913030dcac None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.211797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3811f56-5d4f-4df4-9dab-59913030dcac None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.212118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3811f56-5d4f-4df4-9dab-59913030dcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.239222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c3811f56-5d4f-4df4-9dab-59913030dcac None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:57.240504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c3811f56-5d4f-4df4-9dab-59913030dcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:57.265284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 882/1732] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:57.274885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653460d-3832-49cb-b81b-d4bde1a80a85 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.275600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653460d-3832-49cb-b81b-d4bde1a80a85 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.276054 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653460d-3832-49cb-b81b-d4bde1a80a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.285301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:57.333988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0020830fbb64959ba107dbc4f7f7e3a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:57.339549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0020830fbb64959ba107dbc4f7f7e3a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:57.346052 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8653460d-3832-49cb-b81b-d4bde1a80a85 None None] MFA Rules not processed for user `3a0feaf83ee14071806f36d01c36b635`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:57.349480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8653460d-3832-49cb-b81b-d4bde1a80a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:57.358469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 883/1733] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:42:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:57.359655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653460d-3832-49cb-b81b-d4bde1a80a85 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:57.361706 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8653460d-3832-49cb-b81b-d4bde1a80a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:57.369108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0020830fbb64959ba107dbc4f7f7e3a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:57.388305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653460d-3832-49cb-b81b-d4bde1a80a85 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:57.394704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635479f5-e4d3-4588-8b6b-03097a639c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:57.395635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635479f5-e4d3-4588-8b6b-03097a639c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.396206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635479f5-e4d3-4588-8b6b-03097a639c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.396617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635479f5-e4d3-4588-8b6b-03097a639c7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.397408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635479f5-e4d3-4588-8b6b-03097a639c7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:57.397977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635479f5-e4d3-4588-8b6b-03097a639c7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:57.400080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635479f5-e4d3-4588-8b6b-03097a639c7f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:57.424213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 884/1734] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:57.426594 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635479f5-e4d3-4588-8b6b-03097a639c7f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:57.430676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-635479f5-e4d3-4588-8b6b-03097a639c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:57.449447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635479f5-e4d3-4588-8b6b-03097a639c7f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:57.454668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a950d8d-f758-4e2e-8936-4ca9c83d16c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:57.456380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a950d8d-f758-4e2e-8936-4ca9c83d16c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.457008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a950d8d-f758-4e2e-8936-4ca9c83d16c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.458499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a950d8d-f758-4e2e-8936-4ca9c83d16c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.470872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a950d8d-f758-4e2e-8936-4ca9c83d16c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:57.474763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a950d8d-f758-4e2e-8936-4ca9c83d16c7 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:57.492274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 885/1735] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:57.499337 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a950d8d-f758-4e2e-8936-4ca9c83d16c7 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:57.502329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0020830fbb64959ba107dbc4f7f7e3a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:57.503748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0020830fbb64959ba107dbc4f7f7e3a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:57.532313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 851/1736] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:56 2022] DELETE /identity/v3/users/f0020830fbb64959ba107dbc4f7f7e3a => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:57.535481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a950d8d-f758-4e2e-8936-4ca9c83d16c7 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:57.616599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dab23-f1dc-45a1-adb7-dd08e730b522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:57.617384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dab23-f1dc-45a1-adb7-dd08e730b522 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.617802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dab23-f1dc-45a1-adb7-dd08e730b522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.618215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dab23-f1dc-45a1-adb7-dd08e730b522 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.619212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066dab23-f1dc-45a1-adb7-dd08e730b522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:57.621957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066dab23-f1dc-45a1-adb7-dd08e730b522 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:57.657535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:57.659117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:57.674084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86e28-470c-4409-a098-0d051c3bd39a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:57.680794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:57.681594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.688552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.688552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] PATH_INFO: `/v3/projects/b47daea4aa234222a65a1e0c934fe745` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.688552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b47daea4aa234222a65a1e0c934fe745)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:57.688552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:57.714080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:57.717467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 886/1737] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:57 2022] POST /identity/v3/projects => generated 365 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:42:57.722569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dab23-f1dc-45a1-adb7-dd08e730b522 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:57.850431 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b47daea4aa234222a65a1e0c934fe745', 'request_id': 'req-46741ae7-836f-451b-9542-9cd9ec130c28'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:57.856725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dab23-f1dc-45a1-adb7-dd08e730b522 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:57.868806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b47daea4aa234222a65a1e0c934fe745', 'request_id': 'req-46741ae7-836f-451b-9542-9cd9ec130c28'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:57.952579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d312aed-a55d-4793-b7c2-0600a95ec81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:57.953234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d312aed-a55d-4793-b7c2-0600a95ec81c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:57.953515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d312aed-a55d-4793-b7c2-0600a95ec81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:57.953787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d312aed-a55d-4793-b7c2-0600a95ec81c admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:57.954472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d312aed-a55d-4793-b7c2-0600a95ec81c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:57.956656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d312aed-a55d-4793-b7c2-0600a95ec81c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:57.979792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b47daea4aa234222a65a1e0c934fe745', 'request_id': 'req-46741ae7-836f-451b-9542-9cd9ec130c28'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:57.997864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] The token cache is being invalidate because project b47daea4aa234222a65a1e0c934fe745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:42:57.998247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:57.999143 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.001539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 852/1738] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:57 2022] DELETE /identity/v3/projects/b47daea4aa234222a65a1e0c934fe745 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:42:58.005502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:58.007312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:58.022338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46741ae7-836f-451b-9542-9cd9ec130c28 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:58.031739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:58.032706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:58.033165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:58.033635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] PATH_INFO: `/v3/users/3334c5fa7b07462a9199df67c5f179ee` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:58.036182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3334c5fa7b07462a9199df67c5f179ee)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:58.038707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:58.090562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 887/1739] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:42:57 2022] POST /identity/v3/users => generated 283 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:42:58.095212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d312aed-a55d-4793-b7c2-0600a95ec81c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:58.098671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7d312aed-a55d-4793-b7c2-0600a95ec81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:58.149295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d312aed-a55d-4793-b7c2-0600a95ec81c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:58.169051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bce654-6242-4e15-9882-f75f08b7e683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:58.169729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bce654-6242-4e15-9882-f75f08b7e683 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:58.170014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bce654-6242-4e15-9882-f75f08b7e683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:58.174279 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bce654-6242-4e15-9882-f75f08b7e683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:58.175149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bce654-6242-4e15-9882-f75f08b7e683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:58.176937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bce654-6242-4e15-9882-f75f08b7e683 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:58.201059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 888/1740] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:58.202096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bce654-6242-4e15-9882-f75f08b7e683 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:58.203896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-71bce654-6242-4e15-9882-f75f08b7e683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:58.293824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bce654-6242-4e15-9882-f75f08b7e683 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:58.327857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf11a5a-e8c6-4083-a1f4-4b5627d2bc81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:58.338596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf11a5a-e8c6-4083-a1f4-4b5627d2bc81 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:58.339172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf11a5a-e8c6-4083-a1f4-4b5627d2bc81 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:58.341472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf11a5a-e8c6-4083-a1f4-4b5627d2bc81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:58.372984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0bf11a5a-e8c6-4083-a1f4-4b5627d2bc81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:58.392426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3334c5fa7b07462a9199df67c5f179ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.395407 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3334c5fa7b07462a9199df67c5f179ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.419289 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf11a5a-e8c6-4083-a1f4-4b5627d2bc81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:58.422127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf11a5a-e8c6-4083-a1f4-4b5627d2bc81 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:58.439557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3334c5fa7b07462a9199df67c5f179ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.460995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 889/1741] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:58.462039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf11a5a-e8c6-4083-a1f4-4b5627d2bc81 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:58.474649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf11a5a-e8c6-4083-a1f4-4b5627d2bc81 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:58.481605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a55d-8bc2-4443-bac4-18ef033e6985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:58.482396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a55d-8bc2-4443-bac4-18ef033e6985 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:58.482739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a55d-8bc2-4443-bac4-18ef033e6985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:58.483087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a55d-8bc2-4443-bac4-18ef033e6985 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:58.483844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75a55d-8bc2-4443-bac4-18ef033e6985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:58.487172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75a55d-8bc2-4443-bac4-18ef033e6985 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:58.502080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 890/1742] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:42:58.503608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a55d-8bc2-4443-bac4-18ef033e6985 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:58.506077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6b75a55d-8bc2-4443-bac4-18ef033e6985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:58.539543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a55d-8bc2-4443-bac4-18ef033e6985 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:58.552790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:58.553540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:58.553890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:58.555138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] PATH_INFO: `/v3/projects/b9cd119dd400423299db4ac398dd4ca1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:58.558129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9cd119dd400423299db4ac398dd4ca1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:58.564279 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:58.592342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3334c5fa7b07462a9199df67c5f179ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.594879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3334c5fa7b07462a9199df67c5f179ee'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.634774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 853/1743] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:42:58 2022] DELETE /identity/v3/users/3334c5fa7b07462a9199df67c5f179ee => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:58.635857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:58.640897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:58.776460 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9cd119dd400423299db4ac398dd4ca1', 'request_id': 'req-8c55a870-30ac-4dfd-b4bf-145e151bace6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.794813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9cd119dd400423299db4ac398dd4ca1', 'request_id': 'req-8c55a870-30ac-4dfd-b4bf-145e151bace6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.833973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389eb47-b8d2-41c5-a783-344ae7a605a0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:58.907969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9cd119dd400423299db4ac398dd4ca1', 'request_id': 'req-8c55a870-30ac-4dfd-b4bf-145e151bace6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.931914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] The token cache is being invalidate because project b9cd119dd400423299db4ac398dd4ca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:42:58.931914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.932881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-8c55a870-30ac-4dfd-b4bf-145e151bace6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:58.938600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 891/1744] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:58 2022] DELETE /identity/v3/projects/b9cd119dd400423299db4ac398dd4ca1 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:58.939887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae5446-eee4-458d-9137-36472b97969e None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:58.940170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae5446-eee4-458d-9137-36472b97969e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:58.940493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae5446-eee4-458d-9137-36472b97969e None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:58.942864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 892/1745] 10.222.0.26 () {54 vars in 912 bytes} [Tue Oct 4 05:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:42:58.956740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5425-115a-40a2-8143-2e6a37c804ae None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:58.956740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5425-115a-40a2-8143-2e6a37c804ae None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:58.956740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5425-115a-40a2-8143-2e6a37c804ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:58.973682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-94cf5425-115a-40a2-8143-2e6a37c804ae None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:58.974960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-94cf5425-115a-40a2-8143-2e6a37c804ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:58.975182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d2810-05e4-42bc-a837-dbeb786bc476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:58.976112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d2810-05e4-42bc-a837-dbeb786bc476 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:58.976398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d2810-05e4-42bc-a837-dbeb786bc476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:58.976676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d2810-05e4-42bc-a837-dbeb786bc476 admin admin] PATH_INFO: `/v3/projects/b23ecdb982b546868c1f10f2accd4f0f/users/d12e3571f5a743dca7f60b38335ebe46/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:58.998485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 893/1746] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:59.002326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca89db5-e70d-449f-8251-444a46aa439d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:59.004365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca89db5-e70d-449f-8251-444a46aa439d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:59.004665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca89db5-e70d-449f-8251-444a46aa439d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:59.013831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d2810-05e4-42bc-a837-dbeb786bc476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23ecdb982b546868c1f10f2accd4f0f, user_id=d12e3571f5a743dca7f60b38335ebe46, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:59.017808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d2810-05e4-42bc-a837-dbeb786bc476 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:59.073159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9ca89db5-e70d-449f-8251-444a46aa439d None None] MFA Rules not processed for user `3a0feaf83ee14071806f36d01c36b635`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:59.074760 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9ca89db5-e70d-449f-8251-444a46aa439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:59.095561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 854/1747] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:42:58 2022] PUT /identity/v3/projects/b23ecdb982b546868c1f10f2accd4f0f/users/d12e3571f5a743dca7f60b38335ebe46/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:59.096801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d2810-05e4-42bc-a837-dbeb786bc476 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:59.105621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bb9d2810-05e4-42bc-a837-dbeb786bc476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:59.218954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 894/1748] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:42:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:59.224023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b6914-db25-4d4a-a745-cbcaecb8d99c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:59.224310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b6914-db25-4d4a-a745-cbcaecb8d99c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:59.224590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b6914-db25-4d4a-a745-cbcaecb8d99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:59.249686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d2810-05e4-42bc-a837-dbeb786bc476 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:59.254759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:59.255421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:59.255992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:59.256285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] PATH_INFO: `/v3/projects/7070376ed1fd4d289d306c65471b5e95` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:59.257571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7070376ed1fd4d289d306c65471b5e95)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:59.262045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:59.265736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1e1b6914-db25-4d4a-a745-cbcaecb8d99c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:59.267313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1e1b6914-db25-4d4a-a745-cbcaecb8d99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:59.285710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 895/1749] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:59.287611 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e67bb-eb01-4503-9570-c519b0579234 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:59.288031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e67bb-eb01-4503-9570-c519b0579234 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:59.288268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e67bb-eb01-4503-9570-c519b0579234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:59.327793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-607e67bb-eb01-4503-9570-c519b0579234 None None] MFA Rules not processed for user `d12e3571f5a743dca7f60b38335ebe46`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:42:59.329179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-607e67bb-eb01-4503-9570-c519b0579234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:59.426571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 896/1750] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:42:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:42:59.426571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e67bb-eb01-4503-9570-c519b0579234 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:59.430689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-607e67bb-eb01-4503-9570-c519b0579234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:59.445247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7070376ed1fd4d289d306c65471b5e95', 'request_id': 'req-2909ba90-e5ff-4958-b3ac-40d1977be501'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:59.456663 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7070376ed1fd4d289d306c65471b5e95', 'request_id': 'req-2909ba90-e5ff-4958-b3ac-40d1977be501'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:59.561653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7070376ed1fd4d289d306c65471b5e95', 'request_id': 'req-2909ba90-e5ff-4958-b3ac-40d1977be501'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:59.577600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e67bb-eb01-4503-9570-c519b0579234 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:59.581567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] The token cache is being invalidate because project 7070376ed1fd4d289d306c65471b5e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:42:59.582056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:59.585353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:42:59.587876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 855/1751] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:59 2022] DELETE /identity/v3/projects/7070376ed1fd4d289d306c65471b5e95 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:42:59.597961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:59.602487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:59.669516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e6174-8561-4663-a6c4-ffc7faaedd6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:59.670203 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6174-8561-4663-a6c4-ffc7faaedd6d service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:59.670780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6174-8561-4663-a6c4-ffc7faaedd6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:59.670955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6174-8561-4663-a6c4-ffc7faaedd6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:59.673093 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-614e6174-8561-4663-a6c4-ffc7faaedd6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:59.685772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:42:59.688704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e6174-8561-4663-a6c4-ffc7faaedd6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:59.690478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e6174-8561-4663-a6c4-ffc7faaedd6d service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:59.704480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909ba90-e5ff-4958-b3ac-40d1977be501 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:59.708577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 897/1752] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:42:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:42:59.709443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e6174-8561-4663-a6c4-ffc7faaedd6d service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:59.715305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-614e6174-8561-4663-a6c4-ffc7faaedd6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:59.721665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d401a1d-0b80-44e3-b960-7663703b48c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:59.723121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d401a1d-0b80-44e3-b960-7663703b48c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:59.723783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d401a1d-0b80-44e3-b960-7663703b48c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:59.724157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d401a1d-0b80-44e3-b960-7663703b48c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:59.724975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d401a1d-0b80-44e3-b960-7663703b48c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:42:59.725380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d401a1d-0b80-44e3-b960-7663703b48c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:59.727190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d401a1d-0b80-44e3-b960-7663703b48c9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:59.770956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e6174-8561-4663-a6c4-ffc7faaedd6d service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:59.781040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 856/1753] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:42:59.783475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d401a1d-0b80-44e3-b960-7663703b48c9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:42:59.787552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4d401a1d-0b80-44e3-b960-7663703b48c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:42:59.794578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:59.795237 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:59.798752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:59.799293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] PATH_INFO: `/v3/projects/d76cdc535cb848d5afdff253e48c2b7e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:59.801453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d76cdc535cb848d5afdff253e48c2b7e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:59.807883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:59.808438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d401a1d-0b80-44e3-b960-7663703b48c9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:42:59.826116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65656668-015c-4162-b22e-a016e32f5625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:42:59.828362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65656668-015c-4162-b22e-a016e32f5625 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:42:59.828816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65656668-015c-4162-b22e-a016e32f5625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:42:59.829363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65656668-015c-4162-b22e-a016e32f5625 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:42:59.830258 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65656668-015c-4162-b22e-a016e32f5625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:42:59.833219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65656668-015c-4162-b22e-a016e32f5625 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:42:59.989225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd76cdc535cb848d5afdff253e48c2b7e', 'request_id': 'req-398c3852-1706-4f2a-9364-1afc2db8858a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:00.001302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 857/1754] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:42:59 2022] POST /identity/v3/projects => generated 363 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:00.006192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67699914-c997-4eb8-b9d5-3d6b715827ae None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:00.007462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67699914-c997-4eb8-b9d5-3d6b715827ae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:00.007974 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67699914-c997-4eb8-b9d5-3d6b715827ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:00.012085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd76cdc535cb848d5afdff253e48c2b7e', 'request_id': 'req-398c3852-1706-4f2a-9364-1afc2db8858a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:00.076621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-67699914-c997-4eb8-b9d5-3d6b715827ae None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:00.078012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67699914-c997-4eb8-b9d5-3d6b715827ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:00.149176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd76cdc535cb848d5afdff253e48c2b7e', 'request_id': 'req-398c3852-1706-4f2a-9364-1afc2db8858a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:00.167950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] The token cache is being invalidate because project d76cdc535cb848d5afdff253e48c2b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:00.168416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:00.172852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:00.179964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 898/1755] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:42:59 2022] DELETE /identity/v3/projects/d76cdc535cb848d5afdff253e48c2b7e => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:00.180502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:00.187110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:00.251037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:00.262115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 858/1756] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:00.263251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67699914-c997-4eb8-b9d5-3d6b715827ae None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:00.264759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-67699914-c997-4eb8-b9d5-3d6b715827ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:00.280149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67699914-c997-4eb8-b9d5-3d6b715827ae None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:00.286321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbf2a1-a132-49ea-b331-837290064472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:00.287047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbf2a1-a132-49ea-b331-837290064472 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:00.287947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbf2a1-a132-49ea-b331-837290064472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:00.288286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbf2a1-a132-49ea-b331-837290064472 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:00.290871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbf2a1-a132-49ea-b331-837290064472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:00.294025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbf2a1-a132-49ea-b331-837290064472 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:00.400726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 859/1757] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:00 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:00.401370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c3852-1706-4f2a-9364-1afc2db8858a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:00.407308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f48bd-b4b7-479d-8ea1-a77c0026ac74 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:00.407308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f48bd-b4b7-479d-8ea1-a77c0026ac74 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:00.407308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f48bd-b4b7-479d-8ea1-a77c0026ac74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:00.407795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519347b2-63a4-4615-bf1f-004e5e1deecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:00.407795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519347b2-63a4-4615-bf1f-004e5e1deecf admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:00.407795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519347b2-63a4-4615-bf1f-004e5e1deecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:00.408444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519347b2-63a4-4615-bf1f-004e5e1deecf admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:00.409206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519347b2-63a4-4615-bf1f-004e5e1deecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:00.411538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519347b2-63a4-4615-bf1f-004e5e1deecf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:00.450973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-595f48bd-b4b7-479d-8ea1-a77c0026ac74 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:00.452272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-595f48bd-b4b7-479d-8ea1-a77c0026ac74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:00.470417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 860/1758] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:00.471975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc19017-0cca-4631-8f3b-81469092183d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:00.472436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc19017-0cca-4631-8f3b-81469092183d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:00.472836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc19017-0cca-4631-8f3b-81469092183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:00.501330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8cc19017-0cca-4631-8f3b-81469092183d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:00.503548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8cc19017-0cca-4631-8f3b-81469092183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:00.518687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 861/1759] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:00.522348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc19017-0cca-4631-8f3b-81469092183d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:00.534600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc19017-0cca-4631-8f3b-81469092183d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:00.542102 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e3224-a5c4-4b83-bcca-44c22b159bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:00.543043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e3224-a5c4-4b83-bcca-44c22b159bae admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:00.543491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e3224-a5c4-4b83-bcca-44c22b159bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:00.544941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e3224-a5c4-4b83-bcca-44c22b159bae admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:00.545719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e3224-a5c4-4b83-bcca-44c22b159bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:00.547945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e3224-a5c4-4b83-bcca-44c22b159bae admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:00.562245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 899/1760] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:00 2022] POST /identity/v3/projects => generated 357 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:00.565148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e95cec-020f-474f-a626-c4200971d356 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:00.567048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e95cec-020f-474f-a626-c4200971d356 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:00.568037 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e95cec-020f-474f-a626-c4200971d356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:00.582639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 862/1761] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:00.583516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e3224-a5c4-4b83-bcca-44c22b159bae admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:00.602548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-94e95cec-020f-474f-a626-c4200971d356 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:00.604203 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-94e95cec-020f-474f-a626-c4200971d356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:00.604973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e3224-a5c4-4b83-bcca-44c22b159bae admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:00.698167 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b74180-9e5c-4460-8e3a-44e214a98710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:00.698848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b74180-9e5c-4460-8e3a-44e214a98710 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:00.699250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b74180-9e5c-4460-8e3a-44e214a98710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:00.699525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b74180-9e5c-4460-8e3a-44e214a98710 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:00.700261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b74180-9e5c-4460-8e3a-44e214a98710 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:00.700996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b74180-9e5c-4460-8e3a-44e214a98710 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:00.702822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b74180-9e5c-4460-8e3a-44e214a98710 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:00.734662 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 900/1762] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:00.735681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e95cec-020f-474f-a626-c4200971d356 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:00.748139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 863/1763] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:43:00.754746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b74180-9e5c-4460-8e3a-44e214a98710 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:00.765748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e95cec-020f-474f-a626-c4200971d356 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:00.765748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd27ae7-f9c6-489f-a63a-a14a9a8e184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:00.766751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd27ae7-f9c6-489f-a63a-a14a9a8e184c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:00.766860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b74180-9e5c-4460-8e3a-44e214a98710 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:00.767200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd27ae7-f9c6-489f-a63a-a14a9a8e184c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:00.767566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd27ae7-f9c6-489f-a63a-a14a9a8e184c admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:00.768307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd27ae7-f9c6-489f-a63a-a14a9a8e184c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:00.771283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd27ae7-f9c6-489f-a63a-a14a9a8e184c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:00.772991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709ece8-3e7d-4c1a-a613-6e9df0fb0cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:00.773719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709ece8-3e7d-4c1a-a613-6e9df0fb0cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:00.774004 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709ece8-3e7d-4c1a-a613-6e9df0fb0cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:00.774425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709ece8-3e7d-4c1a-a613-6e9df0fb0cb4 admin admin] PATH_INFO: `/v3/projects/2922dacfcca54424a8c57d40b4ac8aac/users/0ceeda6a725842119eb2dd924364b203/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:00.796226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709ece8-3e7d-4c1a-a613-6e9df0fb0cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2922dacfcca54424a8c57d40b4ac8aac, user_id=0ceeda6a725842119eb2dd924364b203, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:00.799778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709ece8-3e7d-4c1a-a613-6e9df0fb0cb4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:00.880542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 864/1764] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:00 2022] PUT /identity/v3/projects/2922dacfcca54424a8c57d40b4ac8aac/users/0ceeda6a725842119eb2dd924364b203/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:00.885527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709ece8-3e7d-4c1a-a613-6e9df0fb0cb4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:00.962352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 901/1765] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:00 2022] POST /identity/v3/users => generated 279 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:00.963262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd27ae7-f9c6-489f-a63a-a14a9a8e184c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:00.978285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd27ae7-f9c6-489f-a63a-a14a9a8e184c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:01.056944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709ece8-3e7d-4c1a-a613-6e9df0fb0cb4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:01.060536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:01.071609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e075da-baa7-4cba-9349-52cc80c54e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:01.072499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e075da-baa7-4cba-9349-52cc80c54e51 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.072873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e075da-baa7-4cba-9349-52cc80c54e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.073222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e075da-baa7-4cba-9349-52cc80c54e51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.073958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e075da-baa7-4cba-9349-52cc80c54e51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:01.079370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e075da-baa7-4cba-9349-52cc80c54e51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:01.081318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e075da-baa7-4cba-9349-52cc80c54e51 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:01.112859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 865/1766] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:43:01.113881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e075da-baa7-4cba-9349-52cc80c54e51 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:01.116925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3ea1f-78d3-4391-9c96-53dde72518ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:01.119143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ea1f-78d3-4391-9c96-53dde72518ca admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.119143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ea1f-78d3-4391-9c96-53dde72518ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.119143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ea1f-78d3-4391-9c96-53dde72518ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.119143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3ea1f-78d3-4391-9c96-53dde72518ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:01.121000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3ea1f-78d3-4391-9c96-53dde72518ca admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:01.143334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e075da-baa7-4cba-9349-52cc80c54e51 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:01.155796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74809ed2-c0af-402c-8ce6-ee85e2a84516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:01.156428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74809ed2-c0af-402c-8ce6-ee85e2a84516 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.156705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74809ed2-c0af-402c-8ce6-ee85e2a84516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.157241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74809ed2-c0af-402c-8ce6-ee85e2a84516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.157899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74809ed2-c0af-402c-8ce6-ee85e2a84516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:01.159646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74809ed2-c0af-402c-8ce6-ee85e2a84516 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:01.175373 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 866/1767] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:01.176643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74809ed2-c0af-402c-8ce6-ee85e2a84516 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:01.188011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74809ed2-c0af-402c-8ce6-ee85e2a84516 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:01.193741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b69d9-bd57-4c61-938c-bd0c99c3379b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:01.194520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b69d9-bd57-4c61-938c-bd0c99c3379b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.196410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b69d9-bd57-4c61-938c-bd0c99c3379b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.196410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b69d9-bd57-4c61-938c-bd0c99c3379b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.196410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b69d9-bd57-4c61-938c-bd0c99c3379b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:01.197273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b69d9-bd57-4c61-938c-bd0c99c3379b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:01.232965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 867/1768] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:01.233632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b69d9-bd57-4c61-938c-bd0c99c3379b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:01.249053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b69d9-bd57-4c61-938c-bd0c99c3379b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:01.258174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77dab7-52ce-4760-9f6c-30150f81933b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:01.258174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77dab7-52ce-4760-9f6c-30150f81933b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.259042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77dab7-52ce-4760-9f6c-30150f81933b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.259295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77dab7-52ce-4760-9f6c-30150f81933b admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.260146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77dab7-52ce-4760-9f6c-30150f81933b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:01.264171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77dab7-52ce-4760-9f6c-30150f81933b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:01.283409 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 902/1769] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:00 2022] POST /identity/v3/projects => generated 373 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:01.284493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3ea1f-78d3-4391-9c96-53dde72518ca admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:01.438430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 868/1770] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:01 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:01.442677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77dab7-52ce-4760-9f6c-30150f81933b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:01.453916 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3ea1f-78d3-4391-9c96-53dde72518ca admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:01.569690 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77dab7-52ce-4760-9f6c-30150f81933b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:01.578033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7349475-eed3-4271-811f-d7bf237ebb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:01.578900 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7349475-eed3-4271-811f-d7bf237ebb2d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.579304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7349475-eed3-4271-811f-d7bf237ebb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.579839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7349475-eed3-4271-811f-d7bf237ebb2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.580660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7349475-eed3-4271-811f-d7bf237ebb2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:01.583508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7349475-eed3-4271-811f-d7bf237ebb2d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:01.599034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 869/1771] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:01.600164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7349475-eed3-4271-811f-d7bf237ebb2d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:01.613612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7349475-eed3-4271-811f-d7bf237ebb2d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:01.615546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2eb2c-d5cd-4de0-957f-81ca482ab1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:01.620792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49391657-2488-49cb-b278-da9e0ee0ca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:01.621648 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49391657-2488-49cb-b278-da9e0ee0ca2d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.622017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49391657-2488-49cb-b278-da9e0ee0ca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.622432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49391657-2488-49cb-b278-da9e0ee0ca2d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.623268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49391657-2488-49cb-b278-da9e0ee0ca2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:01.625697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49391657-2488-49cb-b278-da9e0ee0ca2d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:01.636253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2eb2c-d5cd-4de0-957f-81ca482ab1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.636562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2eb2c-d5cd-4de0-957f-81ca482ab1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.636854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2eb2c-d5cd-4de0-957f-81ca482ab1c9 admin admin] PATH_INFO: `/v3/projects/2922dacfcca54424a8c57d40b4ac8aac/users/0ceeda6a725842119eb2dd924364b203/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.643878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2eb2c-d5cd-4de0-957f-81ca482ab1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2922dacfcca54424a8c57d40b4ac8aac, user_id=0ceeda6a725842119eb2dd924364b203, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:01.651707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2eb2c-d5cd-4de0-957f-81ca482ab1c9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:01.680242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 903/1772] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:01 2022] PUT /identity/v3/projects/2922dacfcca54424a8c57d40b4ac8aac/users/0ceeda6a725842119eb2dd924364b203/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:01.681465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2eb2c-d5cd-4de0-957f-81ca482ab1c9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:01.763462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2eb2c-d5cd-4de0-957f-81ca482ab1c9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:01.768339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da21c49-f891-491d-a0c5-24178c9fc61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:01.768976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da21c49-f891-491d-a0c5-24178c9fc61a admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.769251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da21c49-f891-491d-a0c5-24178c9fc61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.769517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da21c49-f891-491d-a0c5-24178c9fc61a admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.770164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da21c49-f891-491d-a0c5-24178c9fc61a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:01.772146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da21c49-f891-491d-a0c5-24178c9fc61a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:01.799534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 870/1773] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:01 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:01.801026 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49391657-2488-49cb-b278-da9e0ee0ca2d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:01.816245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49391657-2488-49cb-b278-da9e0ee0ca2d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:01.823486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac512371-6694-43de-8bcc-57b73ff9c75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:01.824600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac512371-6694-43de-8bcc-57b73ff9c75c admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.824887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac512371-6694-43de-8bcc-57b73ff9c75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.825164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac512371-6694-43de-8bcc-57b73ff9c75c admin admin] PATH_INFO: `/v3/projects/815d51a0ef7046d78f240ca95c37b862/users/78d4f764f5d34bfda00d4f0bcfe904a0/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.841533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac512371-6694-43de-8bcc-57b73ff9c75c admin admin] RBAC: Authorizing `identity:create_grant(project_id=815d51a0ef7046d78f240ca95c37b862, user_id=78d4f764f5d34bfda00d4f0bcfe904a0, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:01.844261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac512371-6694-43de-8bcc-57b73ff9c75c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:01.933241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 904/1774] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:01 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:01.935075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060fb53-1d3f-46aa-8a8e-5e2bbe6ce659 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:01.935075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060fb53-1d3f-46aa-8a8e-5e2bbe6ce659 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:01.935627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060fb53-1d3f-46aa-8a8e-5e2bbe6ce659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:01.995380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9060fb53-1d3f-46aa-8a8e-5e2bbe6ce659 None None] MFA Rules not processed for user `0ceeda6a725842119eb2dd924364b203`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:01.996766 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9060fb53-1d3f-46aa-8a8e-5e2bbe6ce659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:02.010717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 871/1775] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:01 2022] PUT /identity/v3/projects/815d51a0ef7046d78f240ca95c37b862/users/78d4f764f5d34bfda00d4f0bcfe904a0/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:02.010717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac512371-6694-43de-8bcc-57b73ff9c75c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:02.150421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 905/1776] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:02.151436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060fb53-1d3f-46aa-8a8e-5e2bbe6ce659 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:02.172282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac512371-6694-43de-8bcc-57b73ff9c75c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:02.183107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d2a94-85cc-4da8-9f9c-467e60fddfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:02.183560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d2a94-85cc-4da8-9f9c-467e60fddfae admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.183843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d2a94-85cc-4da8-9f9c-467e60fddfae admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.185164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d2a94-85cc-4da8-9f9c-467e60fddfae admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.185164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d2a94-85cc-4da8-9f9c-467e60fddfae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:02.186625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d2a94-85cc-4da8-9f9c-467e60fddfae admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:02.198871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 872/1777] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:02.202907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d3fcb-a2ae-402b-95ad-6e8b9ed75c3c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.203197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d3fcb-a2ae-402b-95ad-6e8b9ed75c3c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.203573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d3fcb-a2ae-402b-95ad-6e8b9ed75c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.234205 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060fb53-1d3f-46aa-8a8e-5e2bbe6ce659 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:02.239305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d4efb-dd97-4337-ba06-23b79d3362a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:02.239937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d4efb-dd97-4337-ba06-23b79d3362a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.241039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d4efb-dd97-4337-ba06-23b79d3362a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.241039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d4efb-dd97-4337-ba06-23b79d3362a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.241207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d4efb-dd97-4337-ba06-23b79d3362a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:02.242850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d4efb-dd97-4337-ba06-23b79d3362a8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:02.246671 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-893d3fcb-a2ae-402b-95ad-6e8b9ed75c3c None None] MFA Rules not processed for user `78d4f764f5d34bfda00d4f0bcfe904a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:02.254360 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 906/1778] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:02.255215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d4efb-dd97-4337-ba06-23b79d3362a8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:02.259002 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0f5d4efb-dd97-4337-ba06-23b79d3362a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:02.259390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-893d3fcb-a2ae-402b-95ad-6e8b9ed75c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:02.375382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d4efb-dd97-4337-ba06-23b79d3362a8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:02.384208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c203a-ba7a-4efe-ba9e-eb42239211c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:02.385040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c203a-ba7a-4efe-ba9e-eb42239211c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.385522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c203a-ba7a-4efe-ba9e-eb42239211c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.385939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c203a-ba7a-4efe-ba9e-eb42239211c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.403083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 873/1779] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:02.404310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d3fcb-a2ae-402b-95ad-6e8b9ed75c3c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:02.417172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d3fcb-a2ae-402b-95ad-6e8b9ed75c3c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:02.418311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c203a-ba7a-4efe-ba9e-eb42239211c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:02.419827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c203a-ba7a-4efe-ba9e-eb42239211c3 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:02.423131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52ea91-bf0f-4e3e-ab0d-70d3ba67f2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:02.423895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52ea91-bf0f-4e3e-ab0d-70d3ba67f2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.424714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52ea91-bf0f-4e3e-ab0d-70d3ba67f2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.424714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52ea91-bf0f-4e3e-ab0d-70d3ba67f2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.425504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52ea91-bf0f-4e3e-ab0d-70d3ba67f2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:02.434267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52ea91-bf0f-4e3e-ab0d-70d3ba67f2b1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:02.435397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=76,keystone:DELETE=22,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:02.442555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 907/1780] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:02.446864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c203a-ba7a-4efe-ba9e-eb42239211c3 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:02.448082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 874/1781] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:02.452255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52ea91-bf0f-4e3e-ab0d-70d3ba67f2b1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:02.460243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c203a-ba7a-4efe-ba9e-eb42239211c3 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:02.463876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52ea91-bf0f-4e3e-ab0d-70d3ba67f2b1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:02.465452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c5bf5-e0de-43cf-8540-82fa972c93fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:02.466872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c5bf5-e0de-43cf-8540-82fa972c93fb admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.467292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c5bf5-e0de-43cf-8540-82fa972c93fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.467629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c5bf5-e0de-43cf-8540-82fa972c93fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.468354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c5bf5-e0de-43cf-8540-82fa972c93fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:02.470085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c5bf5-e0de-43cf-8540-82fa972c93fb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:02.474734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e772ac-2c82-46c3-a18b-26f214869e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:02.475427 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e772ac-2c82-46c3-a18b-26f214869e32 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.475749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e772ac-2c82-46c3-a18b-26f214869e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.476030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e772ac-2c82-46c3-a18b-26f214869e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.476673 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e772ac-2c82-46c3-a18b-26f214869e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:02.479153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e772ac-2c82-46c3-a18b-26f214869e32 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:02.482830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 908/1782] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:02.484234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c5bf5-e0de-43cf-8540-82fa972c93fb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:02.496553 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c5bf5-e0de-43cf-8540-82fa972c93fb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:02.501657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc33dd-238e-4d1c-8fd4-f67da0a85ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:02.502441 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc33dd-238e-4d1c-8fd4-f67da0a85ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.503325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc33dd-238e-4d1c-8fd4-f67da0a85ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.503665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc33dd-238e-4d1c-8fd4-f67da0a85ed3 admin admin] PATH_INFO: `/v3/projects/ec57e46b298a471b9267e04a93333a9c/users/36d4e8e9ecf94e89a37161032689c4fe/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.520879 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc33dd-238e-4d1c-8fd4-f67da0a85ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec57e46b298a471b9267e04a93333a9c, user_id=36d4e8e9ecf94e89a37161032689c4fe, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:02.526050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc33dd-238e-4d1c-8fd4-f67da0a85ed3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:02.617488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 909/1783] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:02 2022] PUT /identity/v3/projects/ec57e46b298a471b9267e04a93333a9c/users/36d4e8e9ecf94e89a37161032689c4fe/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:02.617488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc33dd-238e-4d1c-8fd4-f67da0a85ed3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:02.618019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 875/1784] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:02 2022] POST /identity/v3/projects => generated 357 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:02.618019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35936a5-302a-4486-82e6-fef7a901a40e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.618019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35936a5-302a-4486-82e6-fef7a901a40e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.618019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35936a5-302a-4486-82e6-fef7a901a40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.636270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f35936a5-302a-4486-82e6-fef7a901a40e None None] MFA Rules not processed for user `537981b1b3a14cb4b10f336ef0362358`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:02.637493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f35936a5-302a-4486-82e6-fef7a901a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:02.712186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 876/1785] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:02.712186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35936a5-302a-4486-82e6-fef7a901a40e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:02.721178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc33dd-238e-4d1c-8fd4-f67da0a85ed3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:02.818084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a955e-5648-44a6-af62-70b44e27ea98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:02.821411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a955e-5648-44a6-af62-70b44e27ea98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.821697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a955e-5648-44a6-af62-70b44e27ea98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.821981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a955e-5648-44a6-af62-70b44e27ea98 admin admin] PATH_INFO: `/v3/projects/7972e78aadad467ca540f5d5a1df5f8a/users/c19f097d833146b68eaf9049c526bc93/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.830771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35936a5-302a-4486-82e6-fef7a901a40e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:02.836818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f208d-3a50-41d9-bed2-21a1b83b0cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:02.837587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f208d-3a50-41d9-bed2-21a1b83b0cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.837874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f208d-3a50-41d9-bed2-21a1b83b0cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.838756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f208d-3a50-41d9-bed2-21a1b83b0cd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.839721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f208d-3a50-41d9-bed2-21a1b83b0cd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:02.841824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a955e-5648-44a6-af62-70b44e27ea98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7972e78aadad467ca540f5d5a1df5f8a, user_id=c19f097d833146b68eaf9049c526bc93, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:02.843185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f208d-3a50-41d9-bed2-21a1b83b0cd9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:02.844270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a955e-5648-44a6-af62-70b44e27ea98 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:02.929823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 910/1786] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:02 2022] PUT /identity/v3/projects/7972e78aadad467ca540f5d5a1df5f8a/users/c19f097d833146b68eaf9049c526bc93/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:02.931223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ddfa9-04c5-451e-8d4c-b1ed5f70cfe8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:02.931569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ddfa9-04c5-451e-8d4c-b1ed5f70cfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:02.931974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ddfa9-04c5-451e-8d4c-b1ed5f70cfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:02.988304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 877/1787] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:02 2022] POST /identity/v3/users => generated 285 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:02.989371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f208d-3a50-41d9-bed2-21a1b83b0cd9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:02.994871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a74ddfa9-04c5-451e-8d4c-b1ed5f70cfe8 None None] MFA Rules not processed for user `36d4e8e9ecf94e89a37161032689c4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:02.995825 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a74ddfa9-04c5-451e-8d4c-b1ed5f70cfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:03.049067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:03.073401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f208d-3a50-41d9-bed2-21a1b83b0cd9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:03.120548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 911/1788] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:03.122571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6e307-0ff2-4d7e-84f6-05e24b102828 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:03.122863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6e307-0ff2-4d7e-84f6-05e24b102828 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:03.123178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6e307-0ff2-4d7e-84f6-05e24b102828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:03.177302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f1f6e307-0ff2-4d7e-84f6-05e24b102828 None None] MFA Rules not processed for user `c19f097d833146b68eaf9049c526bc93`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:03.179039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f1f6e307-0ff2-4d7e-84f6-05e24b102828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:03.185333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250b2b7-f50c-4249-9f43-a73a7204d578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:03.186629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b2b7-f50c-4249-9f43-a73a7204d578 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:03.187020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b2b7-f50c-4249-9f43-a73a7204d578 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:03.187409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b2b7-f50c-4249-9f43-a73a7204d578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:03.198316 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e250b2b7-f50c-4249-9f43-a73a7204d578 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:03.310841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 912/1789] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:03.311924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6e307-0ff2-4d7e-84f6-05e24b102828 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:03.323939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250b2b7-f50c-4249-9f43-a73a7204d578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:03.329474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250b2b7-f50c-4249-9f43-a73a7204d578 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:03.354030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 878/1790] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:03.354928 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250b2b7-f50c-4249-9f43-a73a7204d578 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:03.466298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250b2b7-f50c-4249-9f43-a73a7204d578 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:03.472567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33557c-d41a-47f4-a940-118e65d87162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:03.472567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33557c-d41a-47f4-a940-118e65d87162 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:03.472908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33557c-d41a-47f4-a940-118e65d87162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:03.474390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33557c-d41a-47f4-a940-118e65d87162 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:03.475089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33557c-d41a-47f4-a940-118e65d87162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:03.477230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6e307-0ff2-4d7e-84f6-05e24b102828 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:03.477720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33557c-d41a-47f4-a940-118e65d87162 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:03.483830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b56ace-7968-4c50-a389-fca7609109b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:03.484710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b56ace-7968-4c50-a389-fca7609109b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:03.485068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b56ace-7968-4c50-a389-fca7609109b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:03.485409 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b56ace-7968-4c50-a389-fca7609109b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:03.487572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b56ace-7968-4c50-a389-fca7609109b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:03.489023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b56ace-7968-4c50-a389-fca7609109b1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:03.527513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 913/1791] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:03.528542 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b56ace-7968-4c50-a389-fca7609109b1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:03.557884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b56ace-7968-4c50-a389-fca7609109b1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:03.601085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86963bfe-d280-4cfa-8027-4a508963e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:03.611594 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86963bfe-d280-4cfa-8027-4a508963e778 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:03.611594 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86963bfe-d280-4cfa-8027-4a508963e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:03.613228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86963bfe-d280-4cfa-8027-4a508963e778 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:03.613228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86963bfe-d280-4cfa-8027-4a508963e778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:03.623034 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86963bfe-d280-4cfa-8027-4a508963e778 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:03.641583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 879/1792] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:03 2022] POST /identity/v3/projects => generated 373 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:03.642639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33557c-d41a-47f4-a940-118e65d87162 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:03.757263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33557c-d41a-47f4-a940-118e65d87162 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:03.769646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840b0a2-0c51-40bd-aeb7-6c9becb77d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:03.769646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840b0a2-0c51-40bd-aeb7-6c9becb77d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:03.769646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840b0a2-0c51-40bd-aeb7-6c9becb77d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:03.769646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840b0a2-0c51-40bd-aeb7-6c9becb77d91 admin admin] PATH_INFO: `/v3/projects/bc8809e71ae742f6b105b5fa62e5af6a/users/ecea2ac52ce443a9bd39cb63ea6ef0e7/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:03.788605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840b0a2-0c51-40bd-aeb7-6c9becb77d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8809e71ae742f6b105b5fa62e5af6a, user_id=ecea2ac52ce443a9bd39cb63ea6ef0e7, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:03.793808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840b0a2-0c51-40bd-aeb7-6c9becb77d91 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:03.809683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 914/1793] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:03 2022] POST /identity/v3/projects => generated 379 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:03.809683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86963bfe-d280-4cfa-8027-4a508963e778 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:03.910874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 880/1794] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:03 2022] PUT /identity/v3/projects/bc8809e71ae742f6b105b5fa62e5af6a/users/ecea2ac52ce443a9bd39cb63ea6ef0e7/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:03.911921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840b0a2-0c51-40bd-aeb7-6c9becb77d91 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:03.979999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86963bfe-d280-4cfa-8027-4a508963e778 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:04.097441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840b0a2-0c51-40bd-aeb7-6c9becb77d91 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:04.103891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f571c2-95b3-40a4-8078-769e3dacb12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:04.104662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f571c2-95b3-40a4-8078-769e3dacb12f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.105039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f571c2-95b3-40a4-8078-769e3dacb12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.105413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f571c2-95b3-40a4-8078-769e3dacb12f admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.107189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f571c2-95b3-40a4-8078-769e3dacb12f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:04.109724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f571c2-95b3-40a4-8078-769e3dacb12f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:04.147873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031be5e2-9f95-4b4e-9a08-95eb3c1eefcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:04.148714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031be5e2-9f95-4b4e-9a08-95eb3c1eefcd admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.149069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031be5e2-9f95-4b4e-9a08-95eb3c1eefcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.149408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031be5e2-9f95-4b4e-9a08-95eb3c1eefcd admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.150528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031be5e2-9f95-4b4e-9a08-95eb3c1eefcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:04.154263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031be5e2-9f95-4b4e-9a08-95eb3c1eefcd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:04.206885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 881/1795] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:03 2022] POST /identity/v3/users => generated 290 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:04.210685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f571c2-95b3-40a4-8078-769e3dacb12f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:04.225577 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f571c2-95b3-40a4-8078-769e3dacb12f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:04.231237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42c34c-c7d3-4318-945d-e3289b9a1110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:04.231592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42c34c-c7d3-4318-945d-e3289b9a1110 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.232917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42c34c-c7d3-4318-945d-e3289b9a1110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.233440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42c34c-c7d3-4318-945d-e3289b9a1110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.236367 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42c34c-c7d3-4318-945d-e3289b9a1110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:04.236367 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42c34c-c7d3-4318-945d-e3289b9a1110 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:04.248891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 882/1796] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:04.250349 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42c34c-c7d3-4318-945d-e3289b9a1110 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:04.265073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42c34c-c7d3-4318-945d-e3289b9a1110 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:04.276990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabbbf4-bf5e-43cb-843f-6f50adccc815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:04.276990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabbbf4-bf5e-43cb-843f-6f50adccc815 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.276990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabbbf4-bf5e-43cb-843f-6f50adccc815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.276990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dabbbf4-bf5e-43cb-843f-6f50adccc815 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.287572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabbbf4-bf5e-43cb-843f-6f50adccc815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:04.289497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dabbbf4-bf5e-43cb-843f-6f50adccc815 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:04.312336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 915/1797] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:03 2022] POST /identity/v3/users => generated 293 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:04.314224 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031be5e2-9f95-4b4e-9a08-95eb3c1eefcd admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:04.327848 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031be5e2-9f95-4b4e-9a08-95eb3c1eefcd admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:04.328591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 883/1798] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:04.329601 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabbbf4-bf5e-43cb-843f-6f50adccc815 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:04.336388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0312faf-f78d-4160-8b9a-8cfd2bd740f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:04.337350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0312faf-f78d-4160-8b9a-8cfd2bd740f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.337634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0312faf-f78d-4160-8b9a-8cfd2bd740f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.337907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0312faf-f78d-4160-8b9a-8cfd2bd740f7 admin admin] PATH_INFO: `/v3/projects/bc8809e71ae742f6b105b5fa62e5af6a/users/ecea2ac52ce443a9bd39cb63ea6ef0e7/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.339960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0312faf-f78d-4160-8b9a-8cfd2bd740f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8809e71ae742f6b105b5fa62e5af6a, user_id=ecea2ac52ce443a9bd39cb63ea6ef0e7, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:04.342652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0312faf-f78d-4160-8b9a-8cfd2bd740f7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:04.343927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dabbbf4-bf5e-43cb-843f-6f50adccc815 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:04.349762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd003aa-d1ab-4702-8879-66ccd025fa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:04.350418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd003aa-d1ab-4702-8879-66ccd025fa90 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.350760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd003aa-d1ab-4702-8879-66ccd025fa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.350945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd003aa-d1ab-4702-8879-66ccd025fa90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.351590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd003aa-d1ab-4702-8879-66ccd025fa90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:04.353447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd003aa-d1ab-4702-8879-66ccd025fa90 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:04.363874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 916/1799] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:04 2022] PUT /identity/v3/projects/bc8809e71ae742f6b105b5fa62e5af6a/users/ecea2ac52ce443a9bd39cb63ea6ef0e7/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:04.365329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0312faf-f78d-4160-8b9a-8cfd2bd740f7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:04.385652 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 884/1800] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:04.388692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b1be4-0726-46fd-b4a1-f81558760185 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.388981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b1be4-0726-46fd-b4a1-f81558760185 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.389259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b1be4-0726-46fd-b4a1-f81558760185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.457507 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-103b1be4-0726-46fd-b4a1-f81558760185 None None] MFA Rules not processed for user `ecea2ac52ce443a9bd39cb63ea6ef0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:04.459895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-103b1be4-0726-46fd-b4a1-f81558760185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:04.513405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0312faf-f78d-4160-8b9a-8cfd2bd740f7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:04.519357 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cf8be-8309-4b08-a3ed-33aae0aeac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:04.520038 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf8be-8309-4b08-a3ed-33aae0aeac39 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.520325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf8be-8309-4b08-a3ed-33aae0aeac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.520598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf8be-8309-4b08-a3ed-33aae0aeac39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.522401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cf8be-8309-4b08-a3ed-33aae0aeac39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:04.523704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cf8be-8309-4b08-a3ed-33aae0aeac39 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:04.540161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 917/1801] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:04.541176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cf8be-8309-4b08-a3ed-33aae0aeac39 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:04.554791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cf8be-8309-4b08-a3ed-33aae0aeac39 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:04.559746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:04.560009 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 885/1802] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:04 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:04.561512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab788f23-b984-498b-937f-2281067757f4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.562000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab788f23-b984-498b-937f-2281067757f4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.562330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab788f23-b984-498b-937f-2281067757f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.564840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f8e49-a744-47b6-8a20-3117c7eb98e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:04.566313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f8e49-a744-47b6-8a20-3117c7eb98e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.566829 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f8e49-a744-47b6-8a20-3117c7eb98e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.567557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f8e49-a744-47b6-8a20-3117c7eb98e9 admin admin] PATH_INFO: `/v3/projects/6cbbc66b9a80441781fc83ccf5d82dfa/users/4f4140758779488ba28b54a527031c76/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.585046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f8e49-a744-47b6-8a20-3117c7eb98e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cbbc66b9a80441781fc83ccf5d82dfa, user_id=4f4140758779488ba28b54a527031c76, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:04.587941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f8e49-a744-47b6-8a20-3117c7eb98e9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:04.616008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ab788f23-b984-498b-937f-2281067757f4 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:04.617273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ab788f23-b984-498b-937f-2281067757f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:04.711128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 918/1803] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:04 2022] PUT /identity/v3/projects/6cbbc66b9a80441781fc83ccf5d82dfa/users/4f4140758779488ba28b54a527031c76/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:04.712011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f8e49-a744-47b6-8a20-3117c7eb98e9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:04.726760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 886/1804] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:43:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:04.727998 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab788f23-b984-498b-937f-2281067757f4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:04.819298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab788f23-b984-498b-937f-2281067757f4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:04.822799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f8e49-a744-47b6-8a20-3117c7eb98e9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:04.828529 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8feb6-a14a-4afd-9fc4-c6c62395d653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:04.829283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8feb6-a14a-4afd-9fc4-c6c62395d653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.829643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8feb6-a14a-4afd-9fc4-c6c62395d653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.830022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8feb6-a14a-4afd-9fc4-c6c62395d653 admin admin] PATH_INFO: `/v3/projects/139f897c82d34cdea173e61ed90f8ee4/users/f150f33962a846e6b14ff0906e227802/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.851883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8feb6-a14a-4afd-9fc4-c6c62395d653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=139f897c82d34cdea173e61ed90f8ee4, user_id=f150f33962a846e6b14ff0906e227802, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:04.857185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8feb6-a14a-4afd-9fc4-c6c62395d653 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:04.931126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06abbe1-ccc2-460e-80ff-4b4257a476e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:04.932508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abbe1-ccc2-460e-80ff-4b4257a476e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:04.932508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abbe1-ccc2-460e-80ff-4b4257a476e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:04.932508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abbe1-ccc2-460e-80ff-4b4257a476e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:04.947878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 919/1805] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:04 2022] PUT /identity/v3/projects/139f897c82d34cdea173e61ed90f8ee4/users/f150f33962a846e6b14ff0906e227802/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:04.949241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8feb6-a14a-4afd-9fc4-c6c62395d653 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:05.049069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06abbe1-ccc2-460e-80ff-4b4257a476e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:05.052886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06abbe1-ccc2-460e-80ff-4b4257a476e9 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:05.066850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8feb6-a14a-4afd-9fc4-c6c62395d653 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:05.072729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 887/1806] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:04 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:05.074207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0fde-3c5c-481e-8a50-7be7fd5a46a2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:05.074515 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0fde-3c5c-481e-8a50-7be7fd5a46a2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:05.074775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de0fde-3c5c-481e-8a50-7be7fd5a46a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:05.076930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422a836-6474-451c-b9e5-bd6822bc4a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:05.078925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422a836-6474-451c-b9e5-bd6822bc4a20 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:05.079356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422a836-6474-451c-b9e5-bd6822bc4a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:05.080508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422a836-6474-451c-b9e5-bd6822bc4a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:05.081712 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422a836-6474-451c-b9e5-bd6822bc4a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:05.088353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422a836-6474-451c-b9e5-bd6822bc4a20 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:05.121124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:05.126064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-68de0fde-3c5c-481e-8a50-7be7fd5a46a2 None None] MFA Rules not processed for user `f150f33962a846e6b14ff0906e227802`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:05.128245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-68de0fde-3c5c-481e-8a50-7be7fd5a46a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:05.136536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 920/1807] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:05.141568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422a836-6474-451c-b9e5-bd6822bc4a20 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:05.179864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422a836-6474-451c-b9e5-bd6822bc4a20 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:05.194797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600de45c-5920-49fb-bc1d-91b937c49c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:05.195953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600de45c-5920-49fb-bc1d-91b937c49c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:05.196383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600de45c-5920-49fb-bc1d-91b937c49c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:05.196813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600de45c-5920-49fb-bc1d-91b937c49c70 admin admin] PATH_INFO: `/v3/projects/6cbbc66b9a80441781fc83ccf5d82dfa/users/4f4140758779488ba28b54a527031c76/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:05.202689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600de45c-5920-49fb-bc1d-91b937c49c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cbbc66b9a80441781fc83ccf5d82dfa, user_id=4f4140758779488ba28b54a527031c76, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:05.206441 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600de45c-5920-49fb-bc1d-91b937c49c70 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:05.226353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 921/1808] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:05 2022] PUT /identity/v3/projects/6cbbc66b9a80441781fc83ccf5d82dfa/users/4f4140758779488ba28b54a527031c76/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:05.227965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600de45c-5920-49fb-bc1d-91b937c49c70 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:05.255679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-600de45c-5920-49fb-bc1d-91b937c49c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:05.356641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 888/1809] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:05.360110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe94cf-3ebb-447a-ab6f-8080f6ab6e00 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:05.360302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe94cf-3ebb-447a-ab6f-8080f6ab6e00 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:05.360843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe94cf-3ebb-447a-ab6f-8080f6ab6e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:05.497862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-06fe94cf-3ebb-447a-ab6f-8080f6ab6e00 None None] MFA Rules not processed for user `4f4140758779488ba28b54a527031c76`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:05.506343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-06fe94cf-3ebb-447a-ab6f-8080f6ab6e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:05.526734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600de45c-5920-49fb-bc1d-91b937c49c70 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:05.533101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0f76c-c4f2-45b4-ae27-a8cea64290c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:05.533729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f76c-c4f2-45b4-ae27-a8cea64290c5 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:05.534196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f76c-c4f2-45b4-ae27-a8cea64290c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:05.534850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0f76c-c4f2-45b4-ae27-a8cea64290c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:05.625793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0f76c-c4f2-45b4-ae27-a8cea64290c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:05.642082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0f76c-c4f2-45b4-ae27-a8cea64290c5 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:05.687723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 922/1810] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:43:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:43:05.688976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0f76c-c4f2-45b4-ae27-a8cea64290c5 service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:05.707613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 889/1811] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:05.707613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe94cf-3ebb-447a-ab6f-8080f6ab6e00 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:05.891342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe94cf-3ebb-447a-ab6f-8080f6ab6e00 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:05.901087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0f76c-c4f2-45b4-ae27-a8cea64290c5 service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:05.908201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e562e4d-aef3-48df-8ccd-75815c16a6a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:05.910044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e562e4d-aef3-48df-8ccd-75815c16a6a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:05.910588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e562e4d-aef3-48df-8ccd-75815c16a6a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:05.911003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e562e4d-aef3-48df-8ccd-75815c16a6a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:05.914600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659a352-cb2a-48fc-8c2d-227e7c27a02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:05.915285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659a352-cb2a-48fc-8c2d-227e7c27a02d admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:05.915563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659a352-cb2a-48fc-8c2d-227e7c27a02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:05.915833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659a352-cb2a-48fc-8c2d-227e7c27a02d admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:05.916474 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659a352-cb2a-48fc-8c2d-227e7c27a02d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:05.922843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659a352-cb2a-48fc-8c2d-227e7c27a02d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:06.013763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e562e4d-aef3-48df-8ccd-75815c16a6a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:06.015934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e562e4d-aef3-48df-8ccd-75815c16a6a0 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:06.063966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 890/1812] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:05 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:06.067535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e562e4d-aef3-48df-8ccd-75815c16a6a0 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:06.068597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 923/1813] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:05 2022] POST /identity/v3/projects => generated 379 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:06.072203 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962d8b8-cc57-45d9-9327-80a06396ab04 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:06.074020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962d8b8-cc57-45d9-9327-80a06396ab04 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:06.074179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7e562e4d-aef3-48df-8ccd-75815c16a6a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:06.074327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962d8b8-cc57-45d9-9327-80a06396ab04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:06.124724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e562e4d-aef3-48df-8ccd-75815c16a6a0 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:06.196582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0962d8b8-cc57-45d9-9327-80a06396ab04 None None] MFA Rules not processed for user `3a0feaf83ee14071806f36d01c36b635`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:06.210235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:06.212485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0962d8b8-cc57-45d9-9327-80a06396ab04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:06.341032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c6f3e-29ee-4c7c-a09c-c838505d6b2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:06.356301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c6f3e-29ee-4c7c-a09c-c838505d6b2b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:06.356301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c6f3e-29ee-4c7c-a09c-c838505d6b2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:06.356301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c6f3e-29ee-4c7c-a09c-c838505d6b2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:06.401564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:06.406827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 924/1814] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Oct 4 05:43:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:06.408710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962d8b8-cc57-45d9-9327-80a06396ab04 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:06.574189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c6f3e-29ee-4c7c-a09c-c838505d6b2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:06.576549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c6f3e-29ee-4c7c-a09c-c838505d6b2b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:06.592336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962d8b8-cc57-45d9-9327-80a06396ab04 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:06.597713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 891/1815] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:06.598642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c6f3e-29ee-4c7c-a09c-c838505d6b2b service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:06.600301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-607c6f3e-29ee-4c7c-a09c-c838505d6b2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:06.602977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cef33a-7e8e-43dc-9c23-59656b8ddcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:06.604371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cef33a-7e8e-43dc-9c23-59656b8ddcbc admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:06.604371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cef33a-7e8e-43dc-9c23-59656b8ddcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:06.604371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cef33a-7e8e-43dc-9c23-59656b8ddcbc admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:06.606585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cef33a-7e8e-43dc-9c23-59656b8ddcbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:06.608643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cef33a-7e8e-43dc-9c23-59656b8ddcbc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:06.638726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c6f3e-29ee-4c7c-a09c-c838505d6b2b service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:06.651397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd20a4c-46d7-4ac5-9e2a-90a3642ce96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:06.653768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd20a4c-46d7-4ac5-9e2a-90a3642ce96d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:06.654363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd20a4c-46d7-4ac5-9e2a-90a3642ce96d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:06.654872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd20a4c-46d7-4ac5-9e2a-90a3642ce96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:06.677017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd20a4c-46d7-4ac5-9e2a-90a3642ce96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:06.679618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd20a4c-46d7-4ac5-9e2a-90a3642ce96d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:06.694061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 892/1816] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:06.749798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 925/1817] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:06 2022] POST /identity/v3/users => generated 296 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:06.756580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd20a4c-46d7-4ac5-9e2a-90a3642ce96d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:06.832627 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd20a4c-46d7-4ac5-9e2a-90a3642ce96d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:06.860655 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1366f64-a791-44f4-a46f-e7b145d8bbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:06.863199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1366f64-a791-44f4-a46f-e7b145d8bbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:06.865300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1366f64-a791-44f4-a46f-e7b145d8bbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:06.865674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1366f64-a791-44f4-a46f-e7b145d8bbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:06.870060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1366f64-a791-44f4-a46f-e7b145d8bbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:06.891886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1366f64-a791-44f4-a46f-e7b145d8bbf2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:07.117285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 893/1818] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:07.120769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cef33a-7e8e-43dc-9c23-59656b8ddcbc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:07.143375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cef33a-7e8e-43dc-9c23-59656b8ddcbc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:07.157053 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4cf73-243a-4d26-958f-e572d669fd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:07.158261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4cf73-243a-4d26-958f-e572d669fd2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:07.158543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4cf73-243a-4d26-958f-e572d669fd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:07.158911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4cf73-243a-4d26-958f-e572d669fd2e admin admin] PATH_INFO: `/v3/projects/365784379da441309651d705a9c72e23/users/4faac7b0d4b643bc83c37472f7424c4a/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:07.177764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4cf73-243a-4d26-958f-e572d669fd2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=365784379da441309651d705a9c72e23, user_id=4faac7b0d4b643bc83c37472f7424c4a, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:07.181183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4cf73-243a-4d26-958f-e572d669fd2e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:07.493924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 926/1819] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:07 2022] PUT /identity/v3/projects/365784379da441309651d705a9c72e23/users/4faac7b0d4b643bc83c37472f7424c4a/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:07.499741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1366f64-a791-44f4-a46f-e7b145d8bbf2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:07.517536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4cf73-243a-4d26-958f-e572d669fd2e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:07.538197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1366f64-a791-44f4-a46f-e7b145d8bbf2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:07.697313 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4cf73-243a-4d26-958f-e572d669fd2e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:07.711028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a51a6a-7859-4ca1-99b8-a834664d084d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:07.712945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51a6a-7859-4ca1-99b8-a834664d084d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:07.713350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51a6a-7859-4ca1-99b8-a834664d084d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:07.713675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51a6a-7859-4ca1-99b8-a834664d084d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:07.716583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a51a6a-7859-4ca1-99b8-a834664d084d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:07.726319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a51a6a-7859-4ca1-99b8-a834664d084d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:07.726539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a6991-5127-447a-8251-7125a3870333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:07.727645 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a6991-5127-447a-8251-7125a3870333 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:07.727986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a6991-5127-447a-8251-7125a3870333 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:07.728270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a6991-5127-447a-8251-7125a3870333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:07.809180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 927/1820] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:07.812883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a51a6a-7859-4ca1-99b8-a834664d084d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:07.851831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a51a6a-7859-4ca1-99b8-a834664d084d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:07.869730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b3c97-38ad-4a82-bda0-277327936264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:07.870857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b3c97-38ad-4a82-bda0-277327936264 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:07.872082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b3c97-38ad-4a82-bda0-277327936264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:07.872778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b3c97-38ad-4a82-bda0-277327936264 admin admin] PATH_INFO: `/v3/projects/365784379da441309651d705a9c72e23/users/4faac7b0d4b643bc83c37472f7424c4a/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:07.875961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b3c97-38ad-4a82-bda0-277327936264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365784379da441309651d705a9c72e23, user_id=4faac7b0d4b643bc83c37472f7424c4a, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:07.923216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b3c97-38ad-4a82-bda0-277327936264 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:07.925934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a6991-5127-447a-8251-7125a3870333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:07.930423 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a6991-5127-447a-8251-7125a3870333 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:07.976663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 928/1821] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:07 2022] PUT /identity/v3/projects/365784379da441309651d705a9c72e23/users/4faac7b0d4b643bc83c37472f7424c4a/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:07.984364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 894/1822] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:07.985194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08fc60-2085-44c0-84fa-025c780a8894 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:07.985194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08fc60-2085-44c0-84fa-025c780a8894 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:07.985194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08fc60-2085-44c0-84fa-025c780a8894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:08.036198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8a08fc60-2085-44c0-84fa-025c780a8894 None None] MFA Rules not processed for user `4faac7b0d4b643bc83c37472f7424c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:08.042111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8a08fc60-2085-44c0-84fa-025c780a8894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:08.233787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 929/1823] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:08.571330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a6991-5127-447a-8251-7125a3870333 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:08.584307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a6991-5127-447a-8251-7125a3870333 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:08.720688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:08.781669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a6ca3-ed68-4c94-9bd2-a56b4bbf2550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:08.782663 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a6ca3-ed68-4c94-9bd2-a56b4bbf2550 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:08.783319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a6ca3-ed68-4c94-9bd2-a56b4bbf2550 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:08.783793 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a6ca3-ed68-4c94-9bd2-a56b4bbf2550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:08.802437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a6ca3-ed68-4c94-9bd2-a56b4bbf2550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:08.804822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a6ca3-ed68-4c94-9bd2-a56b4bbf2550 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:08.827925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 895/1824] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:08.985049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08fc60-2085-44c0-84fa-025c780a8894 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:09.001107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08fc60-2085-44c0-84fa-025c780a8894 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:09.007723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451b424-0d7a-4dfa-b0b9-35a393b0c190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:09.007723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451b424-0d7a-4dfa-b0b9-35a393b0c190 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:09.007723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451b424-0d7a-4dfa-b0b9-35a393b0c190 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:09.007723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451b424-0d7a-4dfa-b0b9-35a393b0c190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:09.068564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:09.159306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451b424-0d7a-4dfa-b0b9-35a393b0c190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:09.160377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba34ee9-5612-442d-ac77-b1abd9357511 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:09.160758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba34ee9-5612-442d-ac77-b1abd9357511 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:09.161132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba34ee9-5612-442d-ac77-b1abd9357511 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:09.162626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 896/1825] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:43:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:43:09.164369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451b424-0d7a-4dfa-b0b9-35a393b0c190 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:09.191500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 930/1826] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:09.232425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68cee-c16d-4841-a127-fbeed7517efb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:09.232942 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68cee-c16d-4841-a127-fbeed7517efb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:09.233351 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68cee-c16d-4841-a127-fbeed7517efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:09.298076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-eec68cee-c16d-4841-a127-fbeed7517efb None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:09.301353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eec68cee-c16d-4841-a127-fbeed7517efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:09.512210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 897/1827] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:43:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:09.842491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:10.024676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451b424-0d7a-4dfa-b0b9-35a393b0c190 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:10.159065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:10.168534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451b424-0d7a-4dfa-b0b9-35a393b0c190 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:10.176155 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:10.177571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:10.178407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:10.179225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] PATH_INFO: `/v3/users/d12e3571f5a743dca7f60b38335ebe46` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:10.181619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=d12e3571f5a743dca7f60b38335ebe46)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:10.192163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:10.299955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec68cee-c16d-4841-a127-fbeed7517efb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:10.302707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eec68cee-c16d-4841-a127-fbeed7517efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:10.336492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec68cee-c16d-4841-a127-fbeed7517efb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:10.351886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea5b26-2948-45cf-ad29-6d924363b54f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:10.352701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea5b26-2948-45cf-ad29-6d924363b54f service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:10.353138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea5b26-2948-45cf-ad29-6d924363b54f service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:10.353762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea5b26-2948-45cf-ad29-6d924363b54f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:10.369515 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea5b26-2948-45cf-ad29-6d924363b54f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:10.372269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea5b26-2948-45cf-ad29-6d924363b54f service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:10.388208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 898/1828] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:43:10.511668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd12e3571f5a743dca7f60b38335ebe46'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:10.516726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007695b-b27c-4982-9f54-189e482226f0 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:10.517109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007695b-b27c-4982-9f54-189e482226f0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:10.517585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007695b-b27c-4982-9f54-189e482226f0 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:10.520364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 899/1829] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:43:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:43:10.548718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd12e3571f5a743dca7f60b38335ebe46'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:10.548718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd12e3571f5a743dca7f60b38335ebe46'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:10.577648 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa06ff-1863-4cef-9eda-08931db2f219 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:10.577648 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa06ff-1863-4cef-9eda-08931db2f219 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:10.577648 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa06ff-1863-4cef-9eda-08931db2f219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:10.631459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd12e3571f5a743dca7f60b38335ebe46'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:10.632245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd12e3571f5a743dca7f60b38335ebe46'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:10.640504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-94fa06ff-1863-4cef-9eda-08931db2f219 None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:10.642341 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-94fa06ff-1863-4cef-9eda-08931db2f219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:10.670850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 900/1830] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:43:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:10.684989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 931/1831] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:10 2022] DELETE /identity/v3/users/d12e3571f5a743dca7f60b38335ebe46 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:10.691194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dfd41-0c13-47ed-b4cd-ca253b6782cb None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:10.691726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dfd41-0c13-47ed-b4cd-ca253b6782cb None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:10.692153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dfd41-0c13-47ed-b4cd-ca253b6782cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:10.704876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:10.707976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:10.786829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c62dfd41-0c13-47ed-b4cd-ca253b6782cb None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:10.791112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c62dfd41-0c13-47ed-b4cd-ca253b6782cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:10.887760 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b129b2-4bae-44a2-aa71-fd740fe74bce admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:10.903664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad8803-aa81-4e09-af99-a5f602de0974 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:10.905996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad8803-aa81-4e09-af99-a5f602de0974 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:10.906317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad8803-aa81-4e09-af99-a5f602de0974 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:10.906588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad8803-aa81-4e09-af99-a5f602de0974 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:10.925398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad8803-aa81-4e09-af99-a5f602de0974 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:10.926608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad8803-aa81-4e09-af99-a5f602de0974 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:10.940484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 932/1832] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:10.975019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 901/1833] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:10.984860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad8803-aa81-4e09-af99-a5f602de0974 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:10.986689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-34ad8803-aa81-4e09-af99-a5f602de0974 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:11.110227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad8803-aa81-4e09-af99-a5f602de0974 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:11.139579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1793763-7b54-4f76-9729-a2445e803ce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:11.139579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1793763-7b54-4f76-9729-a2445e803ce3 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:11.139579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1793763-7b54-4f76-9729-a2445e803ce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:11.139579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1793763-7b54-4f76-9729-a2445e803ce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:11.152311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1793763-7b54-4f76-9729-a2445e803ce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:11.154823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1793763-7b54-4f76-9729-a2445e803ce3 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:11.172318 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 933/1834] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:11.371818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:11.918312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:12.331149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dfd41-0c13-47ed-b4cd-ca253b6782cb None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:12.333195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c62dfd41-0c13-47ed-b4cd-ca253b6782cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:12.360636 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dfd41-0c13-47ed-b4cd-ca253b6782cb None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:12.362311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:12.363524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:12.363896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:12.364220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] PATH_INFO: `/v3/projects/b23ecdb982b546868c1f10f2accd4f0f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:12.365457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23ecdb982b546868c1f10f2accd4f0f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:12.367631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:12.567312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23ecdb982b546868c1f10f2accd4f0f', 'request_id': 'req-dec78471-89f6-453d-84e1-301811eb522f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:12.575062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23ecdb982b546868c1f10f2accd4f0f', 'request_id': 'req-dec78471-89f6-453d-84e1-301811eb522f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:12.670799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23ecdb982b546868c1f10f2accd4f0f', 'request_id': 'req-dec78471-89f6-453d-84e1-301811eb522f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:12.686656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] The token cache is being invalidate because project b23ecdb982b546868c1f10f2accd4f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:12.687020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:12.698718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:12.705251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1793763-7b54-4f76-9729-a2445e803ce3 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:12.706419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 902/1835] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:12 2022] DELETE /identity/v3/projects/b23ecdb982b546868c1f10f2accd4f0f => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:12.721004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1793763-7b54-4f76-9729-a2445e803ce3 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:12.851929 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:12.852885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:12.853305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:12.853715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] PATH_INFO: `/v3/users/0ceeda6a725842119eb2dd924364b203` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:12.855096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ceeda6a725842119eb2dd924364b203)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:12.857185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:12.914488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:13.130913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ceeda6a725842119eb2dd924364b203'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:13.171334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ceeda6a725842119eb2dd924364b203'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:13.171334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ceeda6a725842119eb2dd924364b203'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:13.265857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ceeda6a725842119eb2dd924364b203'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:13.266779 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-62506e23-1458-4104-a2c9-f959ee2bea23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ceeda6a725842119eb2dd924364b203'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:13.298934 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 934/1836] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:12 2022] DELETE /identity/v3/users/0ceeda6a725842119eb2dd924364b203 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:13.557209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:13.696693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:14.561304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:14.563310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:14.579412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec78471-89f6-453d-84e1-301811eb522f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:14.653554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:14.654330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:14.654720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:14.655114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] PATH_INFO: `/v3/projects/2922dacfcca54424a8c57d40b4ac8aac` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:14.656351 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2922dacfcca54424a8c57d40b4ac8aac)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:14.658337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:14.765586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2922dacfcca54424a8c57d40b4ac8aac', 'request_id': 'req-363022cc-1d0e-44ad-a011-17fef3b20d49'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:14.772450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2922dacfcca54424a8c57d40b4ac8aac', 'request_id': 'req-363022cc-1d0e-44ad-a011-17fef3b20d49'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:14.858962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2922dacfcca54424a8c57d40b4ac8aac', 'request_id': 'req-363022cc-1d0e-44ad-a011-17fef3b20d49'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:14.870604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] The token cache is being invalidate because project 2922dacfcca54424a8c57d40b4ac8aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:14.870892 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:14.871513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-363022cc-1d0e-44ad-a011-17fef3b20d49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:14.873021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 903/1837] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:14 2022] DELETE /identity/v3/projects/2922dacfcca54424a8c57d40b4ac8aac => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:14.878634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0af6dc-2bd4-4999-bcc8-a73f020af5ce None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:14.879129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0af6dc-2bd4-4999-bcc8-a73f020af5ce None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:14.879232 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0af6dc-2bd4-4999-bcc8-a73f020af5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:14.898903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fe0af6dc-2bd4-4999-bcc8-a73f020af5ce None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:14.900102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fe0af6dc-2bd4-4999-bcc8-a73f020af5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:14.960582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 935/1838] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:14.968443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea5883-da4f-4939-8fea-60bfc3bad31f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:14.968729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea5883-da4f-4939-8fea-60bfc3bad31f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:14.969002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea5883-da4f-4939-8fea-60bfc3bad31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:14.989044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e0ea5883-da4f-4939-8fea-60bfc3bad31f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:14.990265 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e0ea5883-da4f-4939-8fea-60bfc3bad31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:14.997999 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 904/1839] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:15.008257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0af6dc-2bd4-4999-bcc8-a73f020af5ce None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:15.011653 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0af6dc-2bd4-4999-bcc8-a73f020af5ce None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:15.016312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51941a0e-1643-4962-805b-84f7156cb653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:15.016903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941a0e-1643-4962-805b-84f7156cb653 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:15.017177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941a0e-1643-4962-805b-84f7156cb653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:15.017449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51941a0e-1643-4962-805b-84f7156cb653 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:15.018116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51941a0e-1643-4962-805b-84f7156cb653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:15.018446 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51941a0e-1643-4962-805b-84f7156cb653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:15.020033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51941a0e-1643-4962-805b-84f7156cb653 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:15.027718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:15.031163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 936/1840] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:43:15.035783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea5883-da4f-4939-8fea-60bfc3bad31f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:15.044634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea5883-da4f-4939-8fea-60bfc3bad31f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:15.049816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c158d-a6ec-4053-829b-265931da67cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:15.050433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c158d-a6ec-4053-829b-265931da67cb admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:15.050710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c158d-a6ec-4053-829b-265931da67cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:15.050980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c158d-a6ec-4053-829b-265931da67cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:15.051638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c158d-a6ec-4053-829b-265931da67cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:15.053511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c158d-a6ec-4053-829b-265931da67cb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:15.136940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 905/1841] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:15 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:15.139601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:15.140992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51941a0e-1643-4962-805b-84f7156cb653 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:15.151086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51941a0e-1643-4962-805b-84f7156cb653 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:15.206414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea9a6b-3615-4430-8deb-5900111713d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:15.207019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea9a6b-3615-4430-8deb-5900111713d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:15.207294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea9a6b-3615-4430-8deb-5900111713d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:15.207562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea9a6b-3615-4430-8deb-5900111713d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:15.208181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea9a6b-3615-4430-8deb-5900111713d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:15.210220 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea9a6b-3615-4430-8deb-5900111713d4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:15.302928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 937/1842] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:15 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:15.307247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c158d-a6ec-4053-829b-265931da67cb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:15.318444 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c158d-a6ec-4053-829b-265931da67cb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:15.323244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8db4f9-6386-48ba-98fe-b5669ce8a485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:15.324008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db4f9-6386-48ba-98fe-b5669ce8a485 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:15.324356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db4f9-6386-48ba-98fe-b5669ce8a485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:15.325207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db4f9-6386-48ba-98fe-b5669ce8a485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:15.325746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8db4f9-6386-48ba-98fe-b5669ce8a485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:15.327396 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8db4f9-6386-48ba-98fe-b5669ce8a485 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:15.343016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 906/1843] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:15.344179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea9a6b-3615-4430-8deb-5900111713d4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:15.354916 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea9a6b-3615-4430-8deb-5900111713d4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:15.359614 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c27dee-06d3-4229-9855-e8bc251bd201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:15.360219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c27dee-06d3-4229-9855-e8bc251bd201 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:15.360497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c27dee-06d3-4229-9855-e8bc251bd201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:15.360769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c27dee-06d3-4229-9855-e8bc251bd201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:15.361441 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c27dee-06d3-4229-9855-e8bc251bd201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:15.363458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c27dee-06d3-4229-9855-e8bc251bd201 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:15.376626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 938/1844] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:15.377839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8db4f9-6386-48ba-98fe-b5669ce8a485 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:15.398685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8db4f9-6386-48ba-98fe-b5669ce8a485 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:15.404361 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e245c5-ca58-4131-a8f4-06a735408440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:15.405146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e245c5-ca58-4131-a8f4-06a735408440 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:15.405429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e245c5-ca58-4131-a8f4-06a735408440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:15.405704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e245c5-ca58-4131-a8f4-06a735408440 admin admin] PATH_INFO: `/v3/projects/4a8f7a34677d45178518b22328f29c76/users/940a08e4506642e98c75cc2e2dabc05c/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:15.431145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e245c5-ca58-4131-a8f4-06a735408440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8f7a34677d45178518b22328f29c76, user_id=940a08e4506642e98c75cc2e2dabc05c, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:15.434307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e245c5-ca58-4131-a8f4-06a735408440 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:15.510830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 907/1845] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:15 2022] PUT /identity/v3/projects/4a8f7a34677d45178518b22328f29c76/users/940a08e4506642e98c75cc2e2dabc05c/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:15.521566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc2426-e803-4ab3-9000-b035e4728582 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:15.521849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc2426-e803-4ab3-9000-b035e4728582 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:15.522123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc2426-e803-4ab3-9000-b035e4728582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:15.550675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e4dc2426-e803-4ab3-9000-b035e4728582 None None] MFA Rules not processed for user `940a08e4506642e98c75cc2e2dabc05c`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:15.551873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e4dc2426-e803-4ab3-9000-b035e4728582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:15.616808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 939/1846] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:15.627304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e245c5-ca58-4131-a8f4-06a735408440 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:15.629088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b3e245c5-ca58-4131-a8f4-06a735408440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:15.685530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e245c5-ca58-4131-a8f4-06a735408440 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:15.689458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f81cb7-eda4-4ce2-b3c9-f2a6efeff5a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:15.690597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f81cb7-eda4-4ce2-b3c9-f2a6efeff5a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:15.690870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f81cb7-eda4-4ce2-b3c9-f2a6efeff5a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:15.691137 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f81cb7-eda4-4ce2-b3c9-f2a6efeff5a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:15.700474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f81cb7-eda4-4ce2-b3c9-f2a6efeff5a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:15.701932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f81cb7-eda4-4ce2-b3c9-f2a6efeff5a5 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:15.711960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 908/1847] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:15.844969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:16.096436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:16.548995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:17.213626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:17.288467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:17.735049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:18.357668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:18.442524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:19.566578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc2426-e803-4ab3-9000-b035e4728582 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:19.567947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e4dc2426-e803-4ab3-9000-b035e4728582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:19.669954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc2426-e803-4ab3-9000-b035e4728582 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:19.669954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:19.669954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:19.669954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:19.670749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] PATH_INFO: `/v3/users/c19f097d833146b68eaf9049c526bc93` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:19.670749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c19f097d833146b68eaf9049c526bc93)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:19.670749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:20.248182 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c19f097d833146b68eaf9049c526bc93'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:20.263866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c19f097d833146b68eaf9049c526bc93'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:20.264735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c19f097d833146b68eaf9049c526bc93'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:20.370590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c19f097d833146b68eaf9049c526bc93'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:20.371310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c19f097d833146b68eaf9049c526bc93'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:20.387998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 940/1848] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:19 2022] DELETE /identity/v3/users/c19f097d833146b68eaf9049c526bc93 => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:20.390276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c3791-44ad-490e-ab10-ee712cc5e0ae None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:20.391070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c3791-44ad-490e-ab10-ee712cc5e0ae None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:20.391363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c3791-44ad-490e-ab10-ee712cc5e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:20.413283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ed4c3791-44ad-490e-ab10-ee712cc5e0ae None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:20.414543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed4c3791-44ad-490e-ab10-ee712cc5e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:20.427832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:20.458417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:20.477904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 909/1849] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:20.488351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:20.489733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:20.545915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5b27f-3482-44da-b0ea-945aa659e173 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:20.549626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc55fc-7a3b-4ea6-a779-a70864103cd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:20.550300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc55fc-7a3b-4ea6-a779-a70864103cd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:20.550597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc55fc-7a3b-4ea6-a779-a70864103cd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:20.550865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc55fc-7a3b-4ea6-a779-a70864103cd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:20.560066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc55fc-7a3b-4ea6-a779-a70864103cd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:20.561505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc55fc-7a3b-4ea6-a779-a70864103cd2 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:20.572570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 941/1850] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:21.230663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:21.282913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c3791-44ad-490e-ab10-ee712cc5e0ae None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:21.284331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ed4c3791-44ad-490e-ab10-ee712cc5e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:21.296250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c3791-44ad-490e-ab10-ee712cc5e0ae None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:21.301424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:21.309857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:21.309857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:21.309857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] PATH_INFO: `/v3/projects/7972e78aadad467ca540f5d5a1df5f8a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:21.309857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7972e78aadad467ca540f5d5a1df5f8a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:21.309857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:21.446204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7972e78aadad467ca540f5d5a1df5f8a', 'request_id': 'req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:21.453557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7972e78aadad467ca540f5d5a1df5f8a', 'request_id': 'req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:21.511322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7972e78aadad467ca540f5d5a1df5f8a', 'request_id': 'req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:21.523254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] The token cache is being invalidate because project 7972e78aadad467ca540f5d5a1df5f8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:21.523468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:21.524137 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:21.525666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 910/1851] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:21 2022] DELETE /identity/v3/projects/7972e78aadad467ca540f5d5a1df5f8a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:21.529757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc55fc-7a3b-4ea6-a779-a70864103cd2 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:21.539895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc55fc-7a3b-4ea6-a779-a70864103cd2 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:21.597340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:21.597974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:21.598256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:21.598528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] PATH_INFO: `/v3/users/f150f33962a846e6b14ff0906e227802` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:21.599635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f150f33962a846e6b14ff0906e227802)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:21.601533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:21.852109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f150f33962a846e6b14ff0906e227802'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:21.863892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f150f33962a846e6b14ff0906e227802'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:21.864563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f150f33962a846e6b14ff0906e227802'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:21.972676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f150f33962a846e6b14ff0906e227802'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:21.973304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f150f33962a846e6b14ff0906e227802'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:21.986566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 942/1852] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:21 2022] DELETE /identity/v3/users/f150f33962a846e6b14ff0906e227802 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:22.207625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:22.366497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:22.368109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:22.389083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb886dd4-34ab-48dd-8d16-ea1aa36ab0a6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:22.446844 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:22.447479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:22.447759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:22.448036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] PATH_INFO: `/v3/projects/139f897c82d34cdea173e61ed90f8ee4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:22.449223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] RBAC: Authorizing `identity:delete_project(project_id=139f897c82d34cdea173e61ed90f8ee4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:22.451257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:22.551410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:22.555830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '139f897c82d34cdea173e61ed90f8ee4', 'request_id': 'req-4bf9a90e-2a68-4e82-9a97-6151c813a51b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:22.563651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '139f897c82d34cdea173e61ed90f8ee4', 'request_id': 'req-4bf9a90e-2a68-4e82-9a97-6151c813a51b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:22.648034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '139f897c82d34cdea173e61ed90f8ee4', 'request_id': 'req-4bf9a90e-2a68-4e82-9a97-6151c813a51b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:22.660852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] The token cache is being invalidate because project 139f897c82d34cdea173e61ed90f8ee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:22.661100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:22.662832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:22.664047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 911/1853] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:22 2022] DELETE /identity/v3/projects/139f897c82d34cdea173e61ed90f8ee4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:22.667955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:22.670009 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:22.685036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253abf5-3189-4632-91a7-90fa8f2c3e9a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:22.736351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:22.752745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:22.753605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:22.753881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:22.754178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] PATH_INFO: `/v3/users/4faac7b0d4b643bc83c37472f7424c4a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:22.755385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4faac7b0d4b643bc83c37472f7424c4a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:22.757371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:22.999684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4faac7b0d4b643bc83c37472f7424c4a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:23.006125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4faac7b0d4b643bc83c37472f7424c4a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:23.006773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4faac7b0d4b643bc83c37472f7424c4a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:23.078831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4faac7b0d4b643bc83c37472f7424c4a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:23.079574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-064cf694-3d7e-4dbb-9c6b-62fd285ca0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4faac7b0d4b643bc83c37472f7424c4a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:23.092969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 943/1854] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:22 2022] DELETE /identity/v3/users/4faac7b0d4b643bc83c37472f7424c4a => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:23.665953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:23.667524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:23.692128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9a90e-2a68-4e82-9a97-6151c813a51b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:23.757318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:23.762277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:23.762277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:23.762277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] PATH_INFO: `/v3/projects/365784379da441309651d705a9c72e23` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:23.762277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] RBAC: Authorizing `identity:delete_project(project_id=365784379da441309651d705a9c72e23)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:23.762277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:23.910250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365784379da441309651d705a9c72e23', 'request_id': 'req-6ad1834d-66dd-49aa-beca-174c285d879d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:23.918881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365784379da441309651d705a9c72e23', 'request_id': 'req-6ad1834d-66dd-49aa-beca-174c285d879d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:24.007230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365784379da441309651d705a9c72e23', 'request_id': 'req-6ad1834d-66dd-49aa-beca-174c285d879d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:24.018492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] The token cache is being invalidate because project 365784379da441309651d705a9c72e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:24.018776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:24.019392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6ad1834d-66dd-49aa-beca-174c285d879d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:24.020920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 912/1855] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:23 2022] DELETE /identity/v3/projects/365784379da441309651d705a9c72e23 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:24.026853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cae80a-a2d6-4307-ad21-4c912922d9b3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:24.027311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cae80a-a2d6-4307-ad21-4c912922d9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:24.027423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cae80a-a2d6-4307-ad21-4c912922d9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:24.047837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-08cae80a-a2d6-4307-ad21-4c912922d9b3 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:24.049091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-08cae80a-a2d6-4307-ad21-4c912922d9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:24.110642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 944/1856] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:24.119463 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef299757-f900-446f-a8ab-852ffe154b44 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:24.119791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef299757-f900-446f-a8ab-852ffe154b44 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:24.120065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef299757-f900-446f-a8ab-852ffe154b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:24.139425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ef299757-f900-446f-a8ab-852ffe154b44 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:24.140660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ef299757-f900-446f-a8ab-852ffe154b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:24.148332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 913/1857] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:24.152511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cae80a-a2d6-4307-ad21-4c912922d9b3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:24.162533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cae80a-a2d6-4307-ad21-4c912922d9b3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:24.167044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b08935-074c-4e00-8398-33d012c934c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:24.167639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b08935-074c-4e00-8398-33d012c934c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:24.167915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b08935-074c-4e00-8398-33d012c934c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:24.168189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b08935-074c-4e00-8398-33d012c934c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:24.174229 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b08935-074c-4e00-8398-33d012c934c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:24.174229 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b08935-074c-4e00-8398-33d012c934c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:24.174229 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b08935-074c-4e00-8398-33d012c934c2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:24.184321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 945/1858] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:43:24.189249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef299757-f900-446f-a8ab-852ffe154b44 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:24.205356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef299757-f900-446f-a8ab-852ffe154b44 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:24.210049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fd808-ffdc-4f41-a911-a75367460213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:24.210713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fd808-ffdc-4f41-a911-a75367460213 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:24.210986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fd808-ffdc-4f41-a911-a75367460213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:24.211255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fd808-ffdc-4f41-a911-a75367460213 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:24.211877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fd808-ffdc-4f41-a911-a75367460213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:24.213741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fd808-ffdc-4f41-a911-a75367460213 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:24.354967 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 914/1859] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:24 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:24.355194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b08935-074c-4e00-8398-33d012c934c2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:24.362944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b08935-074c-4e00-8398-33d012c934c2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:24.441220 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:24.454032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e758a8-bd03-4731-a65a-efe05b3416d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:24.454721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e758a8-bd03-4731-a65a-efe05b3416d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:24.454995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e758a8-bd03-4731-a65a-efe05b3416d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:24.455265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e758a8-bd03-4731-a65a-efe05b3416d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:24.456448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e758a8-bd03-4731-a65a-efe05b3416d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:24.458379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e758a8-bd03-4731-a65a-efe05b3416d8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:24.548060 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 946/1860] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:24 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:24.554729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fd808-ffdc-4f41-a911-a75367460213 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:24.565011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fd808-ffdc-4f41-a911-a75367460213 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:24.569573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dadf96-9526-46df-9afa-230cc82cfd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:24.570197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dadf96-9526-46df-9afa-230cc82cfd8c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:24.570479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dadf96-9526-46df-9afa-230cc82cfd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:24.570752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dadf96-9526-46df-9afa-230cc82cfd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:24.571433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dadf96-9526-46df-9afa-230cc82cfd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:24.573000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dadf96-9526-46df-9afa-230cc82cfd8c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:24.583112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 915/1861] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:24.587366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e758a8-bd03-4731-a65a-efe05b3416d8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:24.598867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e758a8-bd03-4731-a65a-efe05b3416d8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:24.603549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e523ea-13e0-4103-91c5-2de749876962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:24.604064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e523ea-13e0-4103-91c5-2de749876962 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:24.604337 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e523ea-13e0-4103-91c5-2de749876962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:24.604609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e523ea-13e0-4103-91c5-2de749876962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:24.605243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e523ea-13e0-4103-91c5-2de749876962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:24.606856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e523ea-13e0-4103-91c5-2de749876962 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:24.618128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 947/1862] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:24.623814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dadf96-9526-46df-9afa-230cc82cfd8c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:24.634567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dadf96-9526-46df-9afa-230cc82cfd8c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:24.639199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094cf0a-c538-4867-a38f-da68c27d03c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:24.639847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094cf0a-c538-4867-a38f-da68c27d03c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:24.640121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094cf0a-c538-4867-a38f-da68c27d03c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:24.640420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094cf0a-c538-4867-a38f-da68c27d03c4 admin admin] PATH_INFO: `/v3/projects/785c3ab021df4c02bc24ff7c6d07a319/users/95de661cc7a94d649a7b72ba0b50165f/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:24.656373 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094cf0a-c538-4867-a38f-da68c27d03c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=785c3ab021df4c02bc24ff7c6d07a319, user_id=95de661cc7a94d649a7b72ba0b50165f, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:24.659084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094cf0a-c538-4867-a38f-da68c27d03c4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:24.754043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:24.764094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 916/1863] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:24 2022] PUT /identity/v3/projects/785c3ab021df4c02bc24ff7c6d07a319/users/95de661cc7a94d649a7b72ba0b50165f/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:24.787504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3355c-ea0e-4ddc-abd8-59d206dbe039 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:24.788119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3355c-ea0e-4ddc-abd8-59d206dbe039 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:24.788635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3355c-ea0e-4ddc-abd8-59d206dbe039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:24.824505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-45f3355c-ea0e-4ddc-abd8-59d206dbe039 None None] MFA Rules not processed for user `95de661cc7a94d649a7b72ba0b50165f`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:24.825790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-45f3355c-ea0e-4ddc-abd8-59d206dbe039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:24.898787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 948/1864] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:24.903260 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094cf0a-c538-4867-a38f-da68c27d03c4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:24.971995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094cf0a-c538-4867-a38f-da68c27d03c4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:24.978915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0a8f6-3ddd-4423-a0b5-8f146f8946aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:24.979532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0a8f6-3ddd-4423-a0b5-8f146f8946aa admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:24.979809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0a8f6-3ddd-4423-a0b5-8f146f8946aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:24.980078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0a8f6-3ddd-4423-a0b5-8f146f8946aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:24.980713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0a8f6-3ddd-4423-a0b5-8f146f8946aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:24.982647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0a8f6-3ddd-4423-a0b5-8f146f8946aa admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:25.902548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 917/1865] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:24 2022] POST /identity/v3/projects => generated 385 bytes in 996 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:25.904231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3355c-ea0e-4ddc-abd8-59d206dbe039 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:25.917809 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3355c-ea0e-4ddc-abd8-59d206dbe039 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:25.975817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95900b2f-1d43-4e8e-9d7c-dd75800f3e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:25.976504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95900b2f-1d43-4e8e-9d7c-dd75800f3e4e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:25.976788 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95900b2f-1d43-4e8e-9d7c-dd75800f3e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:25.977060 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95900b2f-1d43-4e8e-9d7c-dd75800f3e4e admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:25.977693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95900b2f-1d43-4e8e-9d7c-dd75800f3e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:25.979680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95900b2f-1d43-4e8e-9d7c-dd75800f3e4e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:26.069029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 949/1866] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:25 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:26.072606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0a8f6-3ddd-4423-a0b5-8f146f8946aa admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:26.082908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0a8f6-3ddd-4423-a0b5-8f146f8946aa admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:26.088072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b87557-ac87-4325-9107-5c6d744753d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:26.088717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b87557-ac87-4325-9107-5c6d744753d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:26.088997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b87557-ac87-4325-9107-5c6d744753d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:26.089267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b87557-ac87-4325-9107-5c6d744753d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:26.089919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b87557-ac87-4325-9107-5c6d744753d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:26.091586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b87557-ac87-4325-9107-5c6d744753d7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:26.102026 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 918/1867] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:26.106352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95900b2f-1d43-4e8e-9d7c-dd75800f3e4e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:26.116774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95900b2f-1d43-4e8e-9d7c-dd75800f3e4e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:26.122027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3a1d7-2f23-410f-9a60-8e5e629c2841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:26.122027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3a1d7-2f23-410f-9a60-8e5e629c2841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:26.122241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3a1d7-2f23-410f-9a60-8e5e629c2841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:26.122506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3a1d7-2f23-410f-9a60-8e5e629c2841 admin admin] PATH_INFO: `/v3/projects/19b2704415014d1e8ffb0c6e6dfa29a2/users/9c9fc09ee2f04634bf0fa259d46d7bc7/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:26.137620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3a1d7-2f23-410f-9a60-8e5e629c2841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b2704415014d1e8ffb0c6e6dfa29a2, user_id=9c9fc09ee2f04634bf0fa259d46d7bc7, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:26.140342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3a1d7-2f23-410f-9a60-8e5e629c2841 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:26.244292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 950/1868] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:26 2022] PUT /identity/v3/projects/19b2704415014d1e8ffb0c6e6dfa29a2/users/9c9fc09ee2f04634bf0fa259d46d7bc7/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:26.248328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b87557-ac87-4325-9107-5c6d744753d7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:26.260256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b87557-ac87-4325-9107-5c6d744753d7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:26.328036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5231115-bba5-43f7-a01f-e6bea00ac99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:26.329036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5231115-bba5-43f7-a01f-e6bea00ac99c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:26.329512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5231115-bba5-43f7-a01f-e6bea00ac99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:26.329965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5231115-bba5-43f7-a01f-e6bea00ac99c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:26.331101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5231115-bba5-43f7-a01f-e6bea00ac99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:26.333422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5231115-bba5-43f7-a01f-e6bea00ac99c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:26.359218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 919/1869] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:26.367726 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3a1d7-2f23-410f-9a60-8e5e629c2841 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:26.379861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3a1d7-2f23-410f-9a60-8e5e629c2841 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:26.384823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe8203-18a5-4194-b7cc-675a3295a1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:26.385591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe8203-18a5-4194-b7cc-675a3295a1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:26.385876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe8203-18a5-4194-b7cc-675a3295a1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:26.386171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe8203-18a5-4194-b7cc-675a3295a1cc admin admin] PATH_INFO: `/v3/projects/19b2704415014d1e8ffb0c6e6dfa29a2/users/9c9fc09ee2f04634bf0fa259d46d7bc7/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:26.388123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe8203-18a5-4194-b7cc-675a3295a1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b2704415014d1e8ffb0c6e6dfa29a2, user_id=9c9fc09ee2f04634bf0fa259d46d7bc7, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:26.390809 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe8203-18a5-4194-b7cc-675a3295a1cc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:26.402030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 951/1870] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:26 2022] PUT /identity/v3/projects/19b2704415014d1e8ffb0c6e6dfa29a2/users/9c9fc09ee2f04634bf0fa259d46d7bc7/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:26.412802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bb819-d109-4baa-93c3-e6b685d46b51 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:26.413173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bb819-d109-4baa-93c3-e6b685d46b51 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:26.413399 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bb819-d109-4baa-93c3-e6b685d46b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:26.441651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0e9bb819-d109-4baa-93c3-e6b685d46b51 None None] MFA Rules not processed for user `9c9fc09ee2f04634bf0fa259d46d7bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:26.443068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0e9bb819-d109-4baa-93c3-e6b685d46b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:26.515235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 920/1871] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:26.526680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe8203-18a5-4194-b7cc-675a3295a1cc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:26.528467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ecfe8203-18a5-4194-b7cc-675a3295a1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:26.590904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe8203-18a5-4194-b7cc-675a3295a1cc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:26.595301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81922a1a-ea91-4af6-9b8d-0b2b8975e8e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:26.598004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922a1a-ea91-4af6-9b8d-0b2b8975e8e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:26.598004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922a1a-ea91-4af6-9b8d-0b2b8975e8e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:26.598004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81922a1a-ea91-4af6-9b8d-0b2b8975e8e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:26.653517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81922a1a-ea91-4af6-9b8d-0b2b8975e8e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:26.655021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81922a1a-ea91-4af6-9b8d-0b2b8975e8e2 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:26.664429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 952/1872] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:26.836502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:27.036548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:27.340776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9bd61-8040-4c77-ba19-7ef3d314fc7a None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:27.340991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9bd61-8040-4c77-ba19-7ef3d314fc7a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:27.341369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9bd61-8040-4c77-ba19-7ef3d314fc7a None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:27.342666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 921/1873] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:43:27.356681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052cfa8-b0ad-4b31-94de-a75e32dd224e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:27.358459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052cfa8-b0ad-4b31-94de-a75e32dd224e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:27.358459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052cfa8-b0ad-4b31-94de-a75e32dd224e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:27.399757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9052cfa8-b0ad-4b31-94de-a75e32dd224e None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:27.406422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9052cfa8-b0ad-4b31-94de-a75e32dd224e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:27.609332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 953/1874] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:43:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:27.706482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9bd61-8040-4c77-ba19-7ef3d314fc7a None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:27.715709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1db9bd61-8040-4c77-ba19-7ef3d314fc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:27.734658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9bd61-8040-4c77-ba19-7ef3d314fc7a None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:27.740547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cceff-5109-4f3f-8922-870d23d59412 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:27.741160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cceff-5109-4f3f-8922-870d23d59412 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:27.741440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cceff-5109-4f3f-8922-870d23d59412 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:27.741702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cceff-5109-4f3f-8922-870d23d59412 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:27.775058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cceff-5109-4f3f-8922-870d23d59412 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:27.781973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cceff-5109-4f3f-8922-870d23d59412 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:27.824070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 922/1875] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:28.875093 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052cfa8-b0ad-4b31-94de-a75e32dd224e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:28.877300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9052cfa8-b0ad-4b31-94de-a75e32dd224e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:28.976306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:28.987461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052cfa8-b0ad-4b31-94de-a75e32dd224e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:28.993391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:28.994048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:28.994796 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:28.995075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] PATH_INFO: `/v3/users/940a08e4506642e98c75cc2e2dabc05c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:28.997719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=940a08e4506642e98c75cc2e2dabc05c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:28.998987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:29.273626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '940a08e4506642e98c75cc2e2dabc05c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:29.287122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '940a08e4506642e98c75cc2e2dabc05c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:29.287852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '940a08e4506642e98c75cc2e2dabc05c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:29.372861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '940a08e4506642e98c75cc2e2dabc05c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:29.373782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '940a08e4506642e98c75cc2e2dabc05c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:29.388981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 954/1876] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:28 2022] DELETE /identity/v3/users/940a08e4506642e98c75cc2e2dabc05c => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:29.393419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb0c8f-17bd-42b8-8083-c9979a280e9b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:29.393908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb0c8f-17bd-42b8-8083-c9979a280e9b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:29.394115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb0c8f-17bd-42b8-8083-c9979a280e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:29.461803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7efb0c8f-17bd-42b8-8083-c9979a280e9b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:29.463159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7efb0c8f-17bd-42b8-8083-c9979a280e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:29.581222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 923/1877] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:29.590641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:29.593819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:29.663404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fa7c9-e91e-4ddf-9400-56690b831c1c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:29.667495 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d5197-1615-4884-a30a-40a5b45a035f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:29.668671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d5197-1615-4884-a30a-40a5b45a035f service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:29.668671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d5197-1615-4884-a30a-40a5b45a035f service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:29.669513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d5197-1615-4884-a30a-40a5b45a035f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:29.679336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d5197-1615-4884-a30a-40a5b45a035f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:29.680895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d5197-1615-4884-a30a-40a5b45a035f service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:29.691234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 955/1878] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:30.141831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb0c8f-17bd-42b8-8083-c9979a280e9b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:30.144174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7efb0c8f-17bd-42b8-8083-c9979a280e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:30.161640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb0c8f-17bd-42b8-8083-c9979a280e9b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:30.182900 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:30.183821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:30.184250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:30.184526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] PATH_INFO: `/v3/projects/4a8f7a34677d45178518b22328f29c76` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:30.185681 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8f7a34677d45178518b22328f29c76)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:30.187967 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:30.326354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8f7a34677d45178518b22328f29c76', 'request_id': 'req-3a9d9997-453f-4764-a308-d43dde18ba5b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:30.339141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8f7a34677d45178518b22328f29c76', 'request_id': 'req-3a9d9997-453f-4764-a308-d43dde18ba5b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:30.432854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8f7a34677d45178518b22328f29c76', 'request_id': 'req-3a9d9997-453f-4764-a308-d43dde18ba5b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:30.447170 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] The token cache is being invalidate because project 4a8f7a34677d45178518b22328f29c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:30.447691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:30.448353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3a9d9997-453f-4764-a308-d43dde18ba5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:30.450234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 924/1879] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:30 2022] DELETE /identity/v3/projects/4a8f7a34677d45178518b22328f29c76 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:30.454837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b1c62-4188-4568-b3a0-480127ba37c6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:30.455282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b1c62-4188-4568-b3a0-480127ba37c6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:30.455413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b1c62-4188-4568-b3a0-480127ba37c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:30.481681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5a1b1c62-4188-4568-b3a0-480127ba37c6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:30.482923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5a1b1c62-4188-4568-b3a0-480127ba37c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:30.547879 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 956/1880] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:30.556115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ee8ad-6b6d-4911-9e88-109dde71d6d0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:30.556404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ee8ad-6b6d-4911-9e88-109dde71d6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:30.558037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ee8ad-6b6d-4911-9e88-109dde71d6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:30.587059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dc7ee8ad-6b6d-4911-9e88-109dde71d6d0 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:30.588592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dc7ee8ad-6b6d-4911-9e88-109dde71d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:30.598488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 925/1881] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:30.602297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b1c62-4188-4568-b3a0-480127ba37c6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:30.614018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b1c62-4188-4568-b3a0-480127ba37c6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:30.619875 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842da96-640a-4724-9071-14a2ac6e988c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:30.620927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842da96-640a-4724-9071-14a2ac6e988c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:30.621431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842da96-640a-4724-9071-14a2ac6e988c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:30.621844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842da96-640a-4724-9071-14a2ac6e988c admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:30.622889 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842da96-640a-4724-9071-14a2ac6e988c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:30.623402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842da96-640a-4724-9071-14a2ac6e988c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:30.625265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842da96-640a-4724-9071-14a2ac6e988c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:30.648202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 957/1882] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:43:30.653965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ee8ad-6b6d-4911-9e88-109dde71d6d0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:30.665271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ee8ad-6b6d-4911-9e88-109dde71d6d0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:30.680683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35686e0-e6ff-4eea-a275-d4398b3cf313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:30.680683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35686e0-e6ff-4eea-a275-d4398b3cf313 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:30.680683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35686e0-e6ff-4eea-a275-d4398b3cf313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:30.680683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35686e0-e6ff-4eea-a275-d4398b3cf313 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:30.681414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35686e0-e6ff-4eea-a275-d4398b3cf313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:30.682812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35686e0-e6ff-4eea-a275-d4398b3cf313 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:30.837913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 926/1883] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:30 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:30.839103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842da96-640a-4724-9071-14a2ac6e988c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:30.845585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:30.851451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842da96-640a-4724-9071-14a2ac6e988c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:30.909107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c74c31-6d7e-4c3d-b6e4-352280e8a3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:30.909727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c74c31-6d7e-4c3d-b6e4-352280e8a3eb admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:30.910001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c74c31-6d7e-4c3d-b6e4-352280e8a3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:30.910302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c74c31-6d7e-4c3d-b6e4-352280e8a3eb admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:30.910960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c74c31-6d7e-4c3d-b6e4-352280e8a3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:30.912946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c74c31-6d7e-4c3d-b6e4-352280e8a3eb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:31.058595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 958/1884] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:30 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:31.062610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35686e0-e6ff-4eea-a275-d4398b3cf313 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:31.076490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35686e0-e6ff-4eea-a275-d4398b3cf313 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:31.081771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b233a8-ebde-4c03-a1c1-67ca2892c945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:31.082443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b233a8-ebde-4c03-a1c1-67ca2892c945 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:31.082745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b233a8-ebde-4c03-a1c1-67ca2892c945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:31.083016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b233a8-ebde-4c03-a1c1-67ca2892c945 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:31.086463 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b233a8-ebde-4c03-a1c1-67ca2892c945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:31.086463 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b233a8-ebde-4c03-a1c1-67ca2892c945 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:31.095412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 927/1885] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:31.099624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c74c31-6d7e-4c3d-b6e4-352280e8a3eb admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:31.109643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c74c31-6d7e-4c3d-b6e4-352280e8a3eb admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:31.115190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a717e4-9deb-4445-8390-8198fc8bf310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:31.115583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a717e4-9deb-4445-8390-8198fc8bf310 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:31.115859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a717e4-9deb-4445-8390-8198fc8bf310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:31.116159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a717e4-9deb-4445-8390-8198fc8bf310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:31.116790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a717e4-9deb-4445-8390-8198fc8bf310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:31.118403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a717e4-9deb-4445-8390-8198fc8bf310 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:31.129340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 959/1886] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:31.133437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b233a8-ebde-4c03-a1c1-67ca2892c945 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:31.143278 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b233a8-ebde-4c03-a1c1-67ca2892c945 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:31.147805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed6dc7-cd16-40a8-92f0-f0b64ab209d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:31.148450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed6dc7-cd16-40a8-92f0-f0b64ab209d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:31.148728 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed6dc7-cd16-40a8-92f0-f0b64ab209d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:31.149005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed6dc7-cd16-40a8-92f0-f0b64ab209d8 admin admin] PATH_INFO: `/v3/projects/425ade64b07f4f17a39ee859f11d402e/users/8dcd26633d614089b6e3227ca886f3a1/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:31.167950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed6dc7-cd16-40a8-92f0-f0b64ab209d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=425ade64b07f4f17a39ee859f11d402e, user_id=8dcd26633d614089b6e3227ca886f3a1, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:31.170625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed6dc7-cd16-40a8-92f0-f0b64ab209d8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:31.232178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:31.237543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 928/1887] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:31 2022] PUT /identity/v3/projects/425ade64b07f4f17a39ee859f11d402e/users/8dcd26633d614089b6e3227ca886f3a1/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:31.247745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497fc2c-95c0-47fc-ae90-161d8e76a55f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:31.248038 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497fc2c-95c0-47fc-ae90-161d8e76a55f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:31.248312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497fc2c-95c0-47fc-ae90-161d8e76a55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:31.278550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c497fc2c-95c0-47fc-ae90-161d8e76a55f None None] MFA Rules not processed for user `8dcd26633d614089b6e3227ca886f3a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:31.279778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c497fc2c-95c0-47fc-ae90-161d8e76a55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:31.359094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 960/1888] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:31.364026 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed6dc7-cd16-40a8-92f0-f0b64ab209d8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:31.440737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed6dc7-cd16-40a8-92f0-f0b64ab209d8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:31.446086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8e0a0-3ca4-4260-a007-810c1e34f477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:31.446766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8e0a0-3ca4-4260-a007-810c1e34f477 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:31.447068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8e0a0-3ca4-4260-a007-810c1e34f477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:31.447264 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8e0a0-3ca4-4260-a007-810c1e34f477 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:31.448076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8e0a0-3ca4-4260-a007-810c1e34f477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:31.449974 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8e0a0-3ca4-4260-a007-810c1e34f477 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:31.530578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 929/1889] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:31 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:31.534494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497fc2c-95c0-47fc-ae90-161d8e76a55f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:31.559520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497fc2c-95c0-47fc-ae90-161d8e76a55f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:31.623782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429dd10-1d08-4b9c-8132-fe1dd37cb78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:31.624402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429dd10-1d08-4b9c-8132-fe1dd37cb78c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:31.624683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429dd10-1d08-4b9c-8132-fe1dd37cb78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:31.624956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429dd10-1d08-4b9c-8132-fe1dd37cb78c admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:31.625607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429dd10-1d08-4b9c-8132-fe1dd37cb78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:31.628190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429dd10-1d08-4b9c-8132-fe1dd37cb78c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:31.718965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 961/1890] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:31 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:31.723858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8e0a0-3ca4-4260-a007-810c1e34f477 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:31.734261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8e0a0-3ca4-4260-a007-810c1e34f477 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:31.738943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d845d-54af-4956-a189-1b5b683930a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:31.739586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d845d-54af-4956-a189-1b5b683930a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:31.740039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d845d-54af-4956-a189-1b5b683930a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:31.740146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d845d-54af-4956-a189-1b5b683930a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:31.740784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d845d-54af-4956-a189-1b5b683930a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:31.742530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d845d-54af-4956-a189-1b5b683930a2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:31.754366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 930/1891] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:31.763103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429dd10-1d08-4b9c-8132-fe1dd37cb78c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:31.768770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:31.774125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429dd10-1d08-4b9c-8132-fe1dd37cb78c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:31.778937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada7dd4-2902-470d-a070-018349565c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:31.779770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada7dd4-2902-470d-a070-018349565c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:31.780201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada7dd4-2902-470d-a070-018349565c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:31.780472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada7dd4-2902-470d-a070-018349565c3d admin admin] PATH_INFO: `/v3/projects/0ad0a3cb77dd4d2cb06a9804d06adf97/users/d46234a8f0df4634b9d8b377bd3082a0/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:31.796287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada7dd4-2902-470d-a070-018349565c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad0a3cb77dd4d2cb06a9804d06adf97, user_id=d46234a8f0df4634b9d8b377bd3082a0, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:31.798970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada7dd4-2902-470d-a070-018349565c3d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:31.910730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 962/1892] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:31 2022] PUT /identity/v3/projects/0ad0a3cb77dd4d2cb06a9804d06adf97/users/d46234a8f0df4634b9d8b377bd3082a0/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:31.914681 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d845d-54af-4956-a189-1b5b683930a2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:31.925504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d845d-54af-4956-a189-1b5b683930a2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:31.992591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64695f-af47-4afa-aac2-a4e9cde774f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:31.993215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64695f-af47-4afa-aac2-a4e9cde774f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:31.993662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64695f-af47-4afa-aac2-a4e9cde774f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:31.994087 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64695f-af47-4afa-aac2-a4e9cde774f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:31.994926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64695f-af47-4afa-aac2-a4e9cde774f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:32.004569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64695f-af47-4afa-aac2-a4e9cde774f1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:32.015498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 931/1893] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:32.020191 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada7dd4-2902-470d-a070-018349565c3d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:32.030492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada7dd4-2902-470d-a070-018349565c3d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:32.035336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4498175-60b8-4726-b99b-03936766c6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:32.035976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498175-60b8-4726-b99b-03936766c6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:32.036280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498175-60b8-4726-b99b-03936766c6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:32.036556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498175-60b8-4726-b99b-03936766c6e4 admin admin] PATH_INFO: `/v3/projects/0ad0a3cb77dd4d2cb06a9804d06adf97/users/d46234a8f0df4634b9d8b377bd3082a0/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:32.038548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4498175-60b8-4726-b99b-03936766c6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad0a3cb77dd4d2cb06a9804d06adf97, user_id=d46234a8f0df4634b9d8b377bd3082a0, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:32.041186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4498175-60b8-4726-b99b-03936766c6e4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:32.053323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 963/1894] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:32 2022] PUT /identity/v3/projects/0ad0a3cb77dd4d2cb06a9804d06adf97/users/d46234a8f0df4634b9d8b377bd3082a0/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:32.065909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff959623-395c-4a36-bff5-e7d3464ab658 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:32.066247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff959623-395c-4a36-bff5-e7d3464ab658 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:32.066528 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff959623-395c-4a36-bff5-e7d3464ab658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:32.095913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ff959623-395c-4a36-bff5-e7d3464ab658 None None] MFA Rules not processed for user `d46234a8f0df4634b9d8b377bd3082a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:32.097319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff959623-395c-4a36-bff5-e7d3464ab658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:32.168571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:32.179224 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 932/1895] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:32.189977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4498175-60b8-4726-b99b-03936766c6e4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:32.191710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d4498175-60b8-4726-b99b-03936766c6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:32.262975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4498175-60b8-4726-b99b-03936766c6e4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:32.262975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148088bc-b746-484f-90dd-acb1af92e700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:32.263776 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148088bc-b746-484f-90dd-acb1af92e700 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:32.264213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148088bc-b746-484f-90dd-acb1af92e700 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:32.264492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148088bc-b746-484f-90dd-acb1af92e700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:32.322897 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148088bc-b746-484f-90dd-acb1af92e700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:32.324246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148088bc-b746-484f-90dd-acb1af92e700 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:32.333382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 964/1896] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:33.133996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff959623-395c-4a36-bff5-e7d3464ab658 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:33.136938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ff959623-395c-4a36-bff5-e7d3464ab658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:33.160407 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff959623-395c-4a36-bff5-e7d3464ab658 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:33.166656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f65f55-493c-4ff3-adcf-35b9fb574e95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:33.167480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f65f55-493c-4ff3-adcf-35b9fb574e95 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:33.168117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f65f55-493c-4ff3-adcf-35b9fb574e95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:33.168882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f65f55-493c-4ff3-adcf-35b9fb574e95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:33.171656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20f65f55-493c-4ff3-adcf-35b9fb574e95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:33.259687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f65f55-493c-4ff3-adcf-35b9fb574e95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:33.261282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f65f55-493c-4ff3-adcf-35b9fb574e95 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:33.303647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 933/1897] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:36.346463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148088bc-b746-484f-90dd-acb1af92e700 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:36.441024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148088bc-b746-484f-90dd-acb1af92e700 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:36.454240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:36.455197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:36.455197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:36.455613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] PATH_INFO: `/v3/users/8dcd26633d614089b6e3227ca886f3a1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:36.457960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dcd26633d614089b6e3227ca886f3a1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:36.460214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:36.488710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:36.648790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dcd26633d614089b6e3227ca886f3a1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:36.661274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dcd26633d614089b6e3227ca886f3a1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:36.662047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dcd26633d614089b6e3227ca886f3a1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:36.696344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f65f55-493c-4ff3-adcf-35b9fb574e95 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:36.698030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-20f65f55-493c-4ff3-adcf-35b9fb574e95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:36.711087 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f65f55-493c-4ff3-adcf-35b9fb574e95 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:36.715833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:36.716482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:36.716766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:36.717039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] PATH_INFO: `/v3/users/78d4f764f5d34bfda00d4f0bcfe904a0` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:36.718223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] RBAC: Authorizing `identity:delete_user(user_id=78d4f764f5d34bfda00d4f0bcfe904a0)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:36.720535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:36.747222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dcd26633d614089b6e3227ca886f3a1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:36.748177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0315a5ec-7aa2-400b-9f4a-0ad2ef9772f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dcd26633d614089b6e3227ca886f3a1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:36.762249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 965/1898] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:36 2022] DELETE /identity/v3/users/8dcd26633d614089b6e3227ca886f3a1 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:36.767306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ced76-4f95-4a16-879e-5b7351bb5f16 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:36.767617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ced76-4f95-4a16-879e-5b7351bb5f16 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:36.767895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ced76-4f95-4a16-879e-5b7351bb5f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:36.787530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0e7ced76-4f95-4a16-879e-5b7351bb5f16 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:36.788835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0e7ced76-4f95-4a16-879e-5b7351bb5f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:36.858691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 966/1899] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:36.869665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ced76-4f95-4a16-879e-5b7351bb5f16 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:36.871218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0e7ced76-4f95-4a16-879e-5b7351bb5f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:36.945815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ced76-4f95-4a16-879e-5b7351bb5f16 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:36.950076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9a7aa-2c07-4317-b89a-3d8ca8ed46c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:36.952039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9a7aa-2c07-4317-b89a-3d8ca8ed46c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:36.952325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9a7aa-2c07-4317-b89a-3d8ca8ed46c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:36.952598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9a7aa-2c07-4317-b89a-3d8ca8ed46c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:36.965421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9a7aa-2c07-4317-b89a-3d8ca8ed46c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:36.967487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9a7aa-2c07-4317-b89a-3d8ca8ed46c5 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:36.977612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78d4f764f5d34bfda00d4f0bcfe904a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:36.978498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 967/1900] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:43:36.978608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78d4f764f5d34bfda00d4f0bcfe904a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:36.996234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78d4f764f5d34bfda00d4f0bcfe904a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:37.123981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78d4f764f5d34bfda00d4f0bcfe904a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:37.124608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78d4f764f5d34bfda00d4f0bcfe904a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:37.137789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 934/1901] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:36 2022] DELETE /identity/v3/users/78d4f764f5d34bfda00d4f0bcfe904a0 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:37.142346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55be32a-147e-4d8b-b34a-322b9dc4ccaf None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:37.143201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55be32a-147e-4d8b-b34a-322b9dc4ccaf None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:37.143483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55be32a-147e-4d8b-b34a-322b9dc4ccaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:37.164081 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a55be32a-147e-4d8b-b34a-322b9dc4ccaf None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:37.165299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a55be32a-147e-4d8b-b34a-322b9dc4ccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:37.225177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:37.270625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 968/1902] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:37.309334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:37.311498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:37.402287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50f1ba-b148-4978-b90f-13890694011e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:37.419215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97eb082-91b8-460a-bbda-526a4118146e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:37.420190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97eb082-91b8-460a-bbda-526a4118146e service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:37.421080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97eb082-91b8-460a-bbda-526a4118146e service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:37.421368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97eb082-91b8-460a-bbda-526a4118146e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:37.438668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97eb082-91b8-460a-bbda-526a4118146e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:37.440692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97eb082-91b8-460a-bbda-526a4118146e service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:37.451980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 935/1903] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:37.566567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55be32a-147e-4d8b-b34a-322b9dc4ccaf None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:37.569222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a55be32a-147e-4d8b-b34a-322b9dc4ccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:37.587967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55be32a-147e-4d8b-b34a-322b9dc4ccaf None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:37.605790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:37.606609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:37.606728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:37.606999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] PATH_INFO: `/v3/projects/425ade64b07f4f17a39ee859f11d402e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:37.608834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=425ade64b07f4f17a39ee859f11d402e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:37.610746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:37.817828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '425ade64b07f4f17a39ee859f11d402e', 'request_id': 'req-18f3c02e-c37f-4146-b515-39e89b827617'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:37.837873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '425ade64b07f4f17a39ee859f11d402e', 'request_id': 'req-18f3c02e-c37f-4146-b515-39e89b827617'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:37.986969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '425ade64b07f4f17a39ee859f11d402e', 'request_id': 'req-18f3c02e-c37f-4146-b515-39e89b827617'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:37.999568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] The token cache is being invalidate because project 425ade64b07f4f17a39ee859f11d402e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:37.999849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.000500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.002186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 969/1904] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:37 2022] DELETE /identity/v3/projects/425ade64b07f4f17a39ee859f11d402e => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:38.006175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97eb082-91b8-460a-bbda-526a4118146e service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:38.012578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:38.018049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97eb082-91b8-460a-bbda-526a4118146e service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:38.029244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:38.031640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:38.106712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:38.108107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:38.108829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:38.109292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] PATH_INFO: `/v3/users/d46234a8f0df4634b9d8b377bd3082a0` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:38.114018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46234a8f0df4634b9d8b377bd3082a0)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:38.117620 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:38.144676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3c02e-c37f-4146-b515-39e89b827617 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:38.164876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:38.165772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:38.166050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:38.166391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] PATH_INFO: `/v3/projects/815d51a0ef7046d78f240ca95c37b862` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:38.167526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=815d51a0ef7046d78f240ca95c37b862)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:38.169756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:38.419352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46234a8f0df4634b9d8b377bd3082a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.422378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46234a8f0df4634b9d8b377bd3082a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.427395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '815d51a0ef7046d78f240ca95c37b862', 'request_id': 'req-9da02349-dbed-4cc0-a057-66da04b95818'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.435897 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '815d51a0ef7046d78f240ca95c37b862', 'request_id': 'req-9da02349-dbed-4cc0-a057-66da04b95818'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.452855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46234a8f0df4634b9d8b377bd3082a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.515902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '815d51a0ef7046d78f240ca95c37b862', 'request_id': 'req-9da02349-dbed-4cc0-a057-66da04b95818'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.529170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] The token cache is being invalidate because project 815d51a0ef7046d78f240ca95c37b862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:38.529844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.531703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.543422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 970/1905] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:38 2022] DELETE /identity/v3/projects/815d51a0ef7046d78f240ca95c37b862 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:38.544658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:38.546826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:38.585072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:38.592737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46234a8f0df4634b9d8b377bd3082a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.593832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46234a8f0df4634b9d8b377bd3082a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.614081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 936/1906] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:38 2022] DELETE /identity/v3/users/d46234a8f0df4634b9d8b377bd3082a0 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:38.628725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da02349-dbed-4cc0-a057-66da04b95818 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:38.714600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:38.715453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:38.715740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:38.716069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] PATH_INFO: `/v3/users/ecea2ac52ce443a9bd39cb63ea6ef0e7` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:38.718489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecea2ac52ce443a9bd39cb63ea6ef0e7)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:38.720572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:38.951604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecea2ac52ce443a9bd39cb63ea6ef0e7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.973548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecea2ac52ce443a9bd39cb63ea6ef0e7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:38.975283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecea2ac52ce443a9bd39cb63ea6ef0e7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:39.072443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecea2ac52ce443a9bd39cb63ea6ef0e7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:39.074604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-25a038ea-d60f-4c08-9924-2587ae4ff7ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecea2ac52ce443a9bd39cb63ea6ef0e7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:39.100947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 971/1907] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:38 2022] DELETE /identity/v3/users/ecea2ac52ce443a9bd39cb63ea6ef0e7 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:39.196630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:39.198746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:39.231604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb4f56-37fa-47d2-9db9-f33ec85be86d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:39.338485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:39.339162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:39.339698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:39.339973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] PATH_INFO: `/v3/projects/0ad0a3cb77dd4d2cb06a9804d06adf97` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:39.341122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad0a3cb77dd4d2cb06a9804d06adf97)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:39.343338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:39.477757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:39.498376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad0a3cb77dd4d2cb06a9804d06adf97', 'request_id': 'req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:39.506287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad0a3cb77dd4d2cb06a9804d06adf97', 'request_id': 'req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:39.597779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad0a3cb77dd4d2cb06a9804d06adf97', 'request_id': 'req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:39.608747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] The token cache is being invalidate because project 0ad0a3cb77dd4d2cb06a9804d06adf97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:39.609038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:39.609674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:39.611919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 937/1908] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:39 2022] DELETE /identity/v3/projects/0ad0a3cb77dd4d2cb06a9804d06adf97 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:39.616588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5d26f-95d0-469c-abd4-7093fe255e4e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:39.617043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5d26f-95d0-469c-abd4-7093fe255e4e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:39.617127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5d26f-95d0-469c-abd4-7093fe255e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:39.639500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-14a5d26f-95d0-469c-abd4-7093fe255e4e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:39.641315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-14a5d26f-95d0-469c-abd4-7093fe255e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:39.679136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:39.680896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:39.683681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:39.782957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 972/1909] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:39.796199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0eff8b-c057-4092-8d3b-6b072595c40c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:39.798309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0eff8b-c057-4092-8d3b-6b072595c40c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:39.799395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0eff8b-c057-4092-8d3b-6b072595c40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:39.805939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a40db-8e6f-47ad-b7d7-d7801c2eda55 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:39.813497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:39.814171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:39.814509 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:39.814782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] PATH_INFO: `/v3/projects/bc8809e71ae742f6b105b5fa62e5af6a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:39.816325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc8809e71ae742f6b105b5fa62e5af6a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:39.818963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:39.842133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1b0eff8b-c057-4092-8d3b-6b072595c40c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:39.843767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1b0eff8b-c057-4092-8d3b-6b072595c40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:39.859719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 973/1910] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:39.864763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0eff8b-c057-4092-8d3b-6b072595c40c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:39.877880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0eff8b-c057-4092-8d3b-6b072595c40c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:39.885500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f53468-c4cc-4518-b241-2172e643d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:39.887039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f53468-c4cc-4518-b241-2172e643d3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:39.887382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f53468-c4cc-4518-b241-2172e643d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:39.887719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f53468-c4cc-4518-b241-2172e643d3e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:39.888727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f53468-c4cc-4518-b241-2172e643d3e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:39.889096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f53468-c4cc-4518-b241-2172e643d3e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:39.892952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f53468-c4cc-4518-b241-2172e643d3e0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:39.906643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 974/1911] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:43:39.911899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f53468-c4cc-4518-b241-2172e643d3e0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:39.921636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f53468-c4cc-4518-b241-2172e643d3e0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:39.927709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b30fcb-4614-4f2b-9711-fdb424f6d675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:39.928320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b30fcb-4614-4f2b-9711-fdb424f6d675 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:39.928593 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b30fcb-4614-4f2b-9711-fdb424f6d675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:39.928859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b30fcb-4614-4f2b-9711-fdb424f6d675 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:39.929492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b30fcb-4614-4f2b-9711-fdb424f6d675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:39.931422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b30fcb-4614-4f2b-9711-fdb424f6d675 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:39.966831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc8809e71ae742f6b105b5fa62e5af6a', 'request_id': 'req-111ed10d-e7c9-473d-9fe9-2471da577597'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:39.975577 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc8809e71ae742f6b105b5fa62e5af6a', 'request_id': 'req-111ed10d-e7c9-473d-9fe9-2471da577597'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:40.021331 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 975/1912] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:39 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:40.025584 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b30fcb-4614-4f2b-9711-fdb424f6d675 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.081747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc8809e71ae742f6b105b5fa62e5af6a', 'request_id': 'req-111ed10d-e7c9-473d-9fe9-2471da577597'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:40.091587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b30fcb-4614-4f2b-9711-fdb424f6d675 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.096847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] The token cache is being invalidate because project bc8809e71ae742f6b105b5fa62e5af6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:40.097267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:40.098127 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-111ed10d-e7c9-473d-9fe9-2471da577597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:40.098264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f574b-2b7f-4a51-bb09-243ddc61b0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.098948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f574b-2b7f-4a51-bb09-243ddc61b0ef admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.099227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f574b-2b7f-4a51-bb09-243ddc61b0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.099550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f574b-2b7f-4a51-bb09-243ddc61b0ef admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.100183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f574b-2b7f-4a51-bb09-243ddc61b0ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.102303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f574b-2b7f-4a51-bb09-243ddc61b0ef admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.105403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 938/1913] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:39 2022] DELETE /identity/v3/projects/bc8809e71ae742f6b105b5fa62e5af6a => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:40.107791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3332c-25fc-4de6-bb5c-984c10022077 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.108553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3332c-25fc-4de6-bb5c-984c10022077 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.109882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3332c-25fc-4de6-bb5c-984c10022077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.135029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-12f3332c-25fc-4de6-bb5c-984c10022077 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:40.136285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-12f3332c-25fc-4de6-bb5c-984c10022077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:40.145333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 939/1914] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:40.152980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8016e3-b0fd-46c0-942a-6b9e9b3b7cde None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.153268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8016e3-b0fd-46c0-942a-6b9e9b3b7cde None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.153542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8016e3-b0fd-46c0-942a-6b9e9b3b7cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.173244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8a8016e3-b0fd-46c0-942a-6b9e9b3b7cde None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:40.175266 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8a8016e3-b0fd-46c0-942a-6b9e9b3b7cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:40.183519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 940/1915] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:40.187879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8016e3-b0fd-46c0-942a-6b9e9b3b7cde None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.197676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8016e3-b0fd-46c0-942a-6b9e9b3b7cde None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.203516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbde481-0afe-4fb3-b7a4-8b132faec18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.204295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbde481-0afe-4fb3-b7a4-8b132faec18c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.204746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbde481-0afe-4fb3-b7a4-8b132faec18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.205215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbde481-0afe-4fb3-b7a4-8b132faec18c admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.205868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbde481-0afe-4fb3-b7a4-8b132faec18c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:40.206195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbde481-0afe-4fb3-b7a4-8b132faec18c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.207801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbde481-0afe-4fb3-b7a4-8b132faec18c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.224062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 941/1916] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:43:40.224968 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbde481-0afe-4fb3-b7a4-8b132faec18c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.235813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbde481-0afe-4fb3-b7a4-8b132faec18c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.241987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98279c54-9f13-4425-893e-8f6dbce307ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.242680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98279c54-9f13-4425-893e-8f6dbce307ff admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.242957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98279c54-9f13-4425-893e-8f6dbce307ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.243225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98279c54-9f13-4425-893e-8f6dbce307ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.243901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98279c54-9f13-4425-893e-8f6dbce307ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.245772 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98279c54-9f13-4425-893e-8f6dbce307ff admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.257363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 976/1917] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:40 2022] POST /identity/v3/users => generated 291 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:40.260768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f574b-2b7f-4a51-bb09-243ddc61b0ef admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.263401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-988f574b-2b7f-4a51-bb09-243ddc61b0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:40.285744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f574b-2b7f-4a51-bb09-243ddc61b0ef admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.290882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6907799-f985-4589-b9fa-61527281c5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.291880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6907799-f985-4589-b9fa-61527281c5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.291880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6907799-f985-4589-b9fa-61527281c5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.292075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6907799-f985-4589-b9fa-61527281c5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.292676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6907799-f985-4589-b9fa-61527281c5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.294375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6907799-f985-4589-b9fa-61527281c5e1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.309702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 977/1918] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:40.310826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6907799-f985-4589-b9fa-61527281c5e1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.321558 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6907799-f985-4589-b9fa-61527281c5e1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.327552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3ed00-4add-4b53-8374-9f803908bad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.328158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3ed00-4add-4b53-8374-9f803908bad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.328501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3ed00-4add-4b53-8374-9f803908bad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.328780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3ed00-4add-4b53-8374-9f803908bad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.329753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3ed00-4add-4b53-8374-9f803908bad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.331148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3ed00-4add-4b53-8374-9f803908bad1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.346178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 978/1919] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:40.348414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3ed00-4add-4b53-8374-9f803908bad1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.361010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3ed00-4add-4b53-8374-9f803908bad1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.368565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d70133-320c-4c6b-89aa-89258f5bfff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.369937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d70133-320c-4c6b-89aa-89258f5bfff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.370484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d70133-320c-4c6b-89aa-89258f5bfff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.371341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d70133-320c-4c6b-89aa-89258f5bfff8 admin admin] PATH_INFO: `/v3/projects/24bb9c290f79414289887665babfc45b/users/75b743d09e9443feb75f51aa9c378a8b/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.392104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 942/1920] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:40 2022] POST /identity/v3/projects => generated 387 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:40.397337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98279c54-9f13-4425-893e-8f6dbce307ff admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.410385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d70133-320c-4c6b-89aa-89258f5bfff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24bb9c290f79414289887665babfc45b, user_id=75b743d09e9443feb75f51aa9c378a8b, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.415895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d70133-320c-4c6b-89aa-89258f5bfff8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.478071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98279c54-9f13-4425-893e-8f6dbce307ff admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.482858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c3040-0a10-4c20-912e-df4b9ab2f714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.483476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3040-0a10-4c20-912e-df4b9ab2f714 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.483752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3040-0a10-4c20-912e-df4b9ab2f714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.484019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3040-0a10-4c20-912e-df4b9ab2f714 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.484681 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3040-0a10-4c20-912e-df4b9ab2f714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.487195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3040-0a10-4c20-912e-df4b9ab2f714 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.527251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 979/1921] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:40 2022] PUT /identity/v3/projects/24bb9c290f79414289887665babfc45b/users/75b743d09e9443feb75f51aa9c378a8b/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:40.536832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d70133-320c-4c6b-89aa-89258f5bfff8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.536832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-99d70133-320c-4c6b-89aa-89258f5bfff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:40.578994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 943/1922] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:40 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:40.580437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b24383-f568-4e9c-b65a-4107e7b32f67 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.580719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b24383-f568-4e9c-b65a-4107e7b32f67 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.581748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b24383-f568-4e9c-b65a-4107e7b32f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.597983 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d70133-320c-4c6b-89aa-89258f5bfff8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.607530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.608171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.608444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.608728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] PATH_INFO: `/v3/users/95de661cc7a94d649a7b72ba0b50165f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.609849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95de661cc7a94d649a7b72ba0b50165f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.610323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e0b24383-f568-4e9c-b65a-4107e7b32f67 None None] MFA Rules not processed for user `75b743d09e9443feb75f51aa9c378a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:40.612155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e0b24383-f568-4e9c-b65a-4107e7b32f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:40.612505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.679429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 944/1923] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:40.680370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b24383-f568-4e9c-b65a-4107e7b32f67 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.694477 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b24383-f568-4e9c-b65a-4107e7b32f67 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.700249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ab269-7190-4f74-a969-953609f30828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.700898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ab269-7190-4f74-a969-953609f30828 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.701174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ab269-7190-4f74-a969-953609f30828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.701500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ab269-7190-4f74-a969-953609f30828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.702190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ab269-7190-4f74-a969-953609f30828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.703816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ab269-7190-4f74-a969-953609f30828 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.715406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 945/1924] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:40.716386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ab269-7190-4f74-a969-953609f30828 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.718398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-931ab269-7190-4f74-a969-953609f30828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:40.774268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ab269-7190-4f74-a969-953609f30828 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.777597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:40.780463 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d6329-25d2-4d0e-aa7f-5e7e95d8975b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.781085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6329-25d2-4d0e-aa7f-5e7e95d8975b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.781377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6329-25d2-4d0e-aa7f-5e7e95d8975b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.781858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d6329-25d2-4d0e-aa7f-5e7e95d8975b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.783970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95de661cc7a94d649a7b72ba0b50165f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:40.791928 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d6329-25d2-4d0e-aa7f-5e7e95d8975b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.793651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d6329-25d2-4d0e-aa7f-5e7e95d8975b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.796892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95de661cc7a94d649a7b72ba0b50165f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:40.797725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95de661cc7a94d649a7b72ba0b50165f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:40.804053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 946/1925] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:40.804966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d6329-25d2-4d0e-aa7f-5e7e95d8975b service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.821174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d6329-25d2-4d0e-aa7f-5e7e95d8975b service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.821937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c18ba6-c412-42ca-8d17-9f82853969cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.822568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c18ba6-c412-42ca-8d17-9f82853969cd admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.822847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c18ba6-c412-42ca-8d17-9f82853969cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.823116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c18ba6-c412-42ca-8d17-9f82853969cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.823756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c18ba6-c412-42ca-8d17-9f82853969cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.825309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c18ba6-c412-42ca-8d17-9f82853969cd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.835412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 947/1926] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:40.839670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c18ba6-c412-42ca-8d17-9f82853969cd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.841479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-77c18ba6-c412-42ca-8d17-9f82853969cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:40.855221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c18ba6-c412-42ca-8d17-9f82853969cd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:40.860180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fe2aa-aab5-4863-8bfb-054baccaf01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:40.860872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fe2aa-aab5-4863-8bfb-054baccaf01f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.861675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fe2aa-aab5-4863-8bfb-054baccaf01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.861947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fe2aa-aab5-4863-8bfb-054baccaf01f admin admin] PATH_INFO: `/v3/projects/dee0411306364b30b9be059c451fb538/users/42b6c20564564ba185a3b84177c10a81/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.869686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95de661cc7a94d649a7b72ba0b50165f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:40.870529 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-754eecaa-c8ff-4fdd-aa13-6fc743a74c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95de661cc7a94d649a7b72ba0b50165f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:40.879978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fe2aa-aab5-4863-8bfb-054baccaf01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee0411306364b30b9be059c451fb538, user_id=42b6c20564564ba185a3b84177c10a81, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:40.882798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fe2aa-aab5-4863-8bfb-054baccaf01f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:40.886191 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 980/1927] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:40 2022] DELETE /identity/v3/users/95de661cc7a94d649a7b72ba0b50165f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:40.889179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b22216-bcdb-417c-b05a-d01600ff86c0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.889459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b22216-bcdb-417c-b05a-d01600ff86c0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.889734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b22216-bcdb-417c-b05a-d01600ff86c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.911631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a3b22216-bcdb-417c-b05a-d01600ff86c0 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:40.912817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a3b22216-bcdb-417c-b05a-d01600ff86c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:40.962717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 948/1928] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:40 2022] PUT /identity/v3/projects/dee0411306364b30b9be059c451fb538/users/42b6c20564564ba185a3b84177c10a81/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:40.974280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01efd-f1f6-4034-8ffc-d01a7b81a341 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:40.974566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01efd-f1f6-4034-8ffc-d01a7b81a341 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:40.974836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01efd-f1f6-4034-8ffc-d01a7b81a341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:40.977795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 981/1929] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:40.982977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:40.991214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b22216-bcdb-417c-b05a-d01600ff86c0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:40.991214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a3b22216-bcdb-417c-b05a-d01600ff86c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:41.004719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bee01efd-f1f6-4034-8ffc-d01a7b81a341 None None] MFA Rules not processed for user `42b6c20564564ba185a3b84177c10a81`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:41.005983 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bee01efd-f1f6-4034-8ffc-d01a7b81a341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:41.053817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b22216-bcdb-417c-b05a-d01600ff86c0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:41.092857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 949/1930] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:41.094437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee01efd-f1f6-4034-8ffc-d01a7b81a341 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:41.104255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b9a40-6a2e-4713-a15e-15fb235e0c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:41.104873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b9a40-6a2e-4713-a15e-15fb235e0c2b service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:41.105158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b9a40-6a2e-4713-a15e-15fb235e0c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:41.105434 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b9a40-6a2e-4713-a15e-15fb235e0c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:41.169153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee01efd-f1f6-4034-8ffc-d01a7b81a341 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:41.174630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df04521-db18-4ea3-9005-35d78b4d6c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:41.175249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df04521-db18-4ea3-9005-35d78b4d6c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:41.175533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df04521-db18-4ea3-9005-35d78b4d6c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:41.175869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df04521-db18-4ea3-9005-35d78b4d6c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:41.176534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df04521-db18-4ea3-9005-35d78b4d6c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:41.178481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df04521-db18-4ea3-9005-35d78b4d6c98 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:41.186679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b9a40-6a2e-4713-a15e-15fb235e0c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:41.189810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b9a40-6a2e-4713-a15e-15fb235e0c2b service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:41.200212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 982/1931] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:43:41.265800 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 950/1932] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:41 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:41.269962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b9a40-6a2e-4713-a15e-15fb235e0c2b service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:41.280099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b9a40-6a2e-4713-a15e-15fb235e0c2b service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:41.337699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af30385-2ee6-4329-8113-ae2fe1e6dcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:41.338366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af30385-2ee6-4329-8113-ae2fe1e6dcf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:41.338645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af30385-2ee6-4329-8113-ae2fe1e6dcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:41.338919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af30385-2ee6-4329-8113-ae2fe1e6dcf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:41.339605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af30385-2ee6-4329-8113-ae2fe1e6dcf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:41.341546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af30385-2ee6-4329-8113-ae2fe1e6dcf6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:41.413813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 983/1933] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:41 2022] POST /identity/v3/users => generated 300 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:41.417580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df04521-db18-4ea3-9005-35d78b4d6c98 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:41.429307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df04521-db18-4ea3-9005-35d78b4d6c98 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:41.454532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a35e39-d02c-4870-a598-95fd40adfe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:41.456699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a35e39-d02c-4870-a598-95fd40adfe8e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:41.457097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a35e39-d02c-4870-a598-95fd40adfe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:41.457442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a35e39-d02c-4870-a598-95fd40adfe8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:41.458226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a35e39-d02c-4870-a598-95fd40adfe8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:41.462898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a35e39-d02c-4870-a598-95fd40adfe8e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:41.474987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 951/1934] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:41.479215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af30385-2ee6-4329-8113-ae2fe1e6dcf6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:41.490531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af30385-2ee6-4329-8113-ae2fe1e6dcf6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:41.496217 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3d57e-fa43-4ab3-b5f3-4c223efcdc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:41.496883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3d57e-fa43-4ab3-b5f3-4c223efcdc02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:41.497157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3d57e-fa43-4ab3-b5f3-4c223efcdc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:41.497484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3d57e-fa43-4ab3-b5f3-4c223efcdc02 admin admin] PATH_INFO: `/v3/projects/c29a5549e70f436097e2aba9e62dbf5c/users/8338d61d164147f7b8f5f28d90309ca2/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:41.518724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3d57e-fa43-4ab3-b5f3-4c223efcdc02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29a5549e70f436097e2aba9e62dbf5c, user_id=8338d61d164147f7b8f5f28d90309ca2, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:41.518724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3d57e-fa43-4ab3-b5f3-4c223efcdc02 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:41.638672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 984/1935] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:41 2022] PUT /identity/v3/projects/c29a5549e70f436097e2aba9e62dbf5c/users/8338d61d164147f7b8f5f28d90309ca2/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:41.643129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a35e39-d02c-4870-a598-95fd40adfe8e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:41.652920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a35e39-d02c-4870-a598-95fd40adfe8e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:41.670476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3d57e-fa43-4ab3-b5f3-4c223efcdc02 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:41.672303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-90b3d57e-fa43-4ab3-b5f3-4c223efcdc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:41.718955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51640dc2-6a11-4212-9c1a-4aba1472785a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:41.725296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51640dc2-6a11-4212-9c1a-4aba1472785a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:41.725296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51640dc2-6a11-4212-9c1a-4aba1472785a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:41.725296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51640dc2-6a11-4212-9c1a-4aba1472785a admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:41.725296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51640dc2-6a11-4212-9c1a-4aba1472785a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:41.725296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51640dc2-6a11-4212-9c1a-4aba1472785a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:41.732813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 952/1936] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:41.736825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51640dc2-6a11-4212-9c1a-4aba1472785a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:41.744569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3d57e-fa43-4ab3-b5f3-4c223efcdc02 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:41.746334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51640dc2-6a11-4212-9c1a-4aba1472785a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:41.750154 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:41.750837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:41.751116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:41.751452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] PATH_INFO: `/v3/projects/785c3ab021df4c02bc24ff7c6d07a319` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:41.751551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81f19d-06ed-43d2-a3d9-4cb409496ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:41.752337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81f19d-06ed-43d2-a3d9-4cb409496ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:41.752621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81f19d-06ed-43d2-a3d9-4cb409496ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:41.752718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=785c3ab021df4c02bc24ff7c6d07a319)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:41.752941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81f19d-06ed-43d2-a3d9-4cb409496ba3 admin admin] PATH_INFO: `/v3/projects/c29a5549e70f436097e2aba9e62dbf5c/users/8338d61d164147f7b8f5f28d90309ca2/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:41.754861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:41.755333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81f19d-06ed-43d2-a3d9-4cb409496ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29a5549e70f436097e2aba9e62dbf5c, user_id=8338d61d164147f7b8f5f28d90309ca2, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:41.758991 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81f19d-06ed-43d2-a3d9-4cb409496ba3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:41.772234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 953/1937] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:41 2022] PUT /identity/v3/projects/c29a5549e70f436097e2aba9e62dbf5c/users/8338d61d164147f7b8f5f28d90309ca2/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:41.781543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:41.784453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bc7b6-9592-4506-a10a-171ed9207104 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:41.784738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bc7b6-9592-4506-a10a-171ed9207104 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:41.785069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bc7b6-9592-4506-a10a-171ed9207104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:41.811757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-784bc7b6-9592-4506-a10a-171ed9207104 None None] MFA Rules not processed for user `8338d61d164147f7b8f5f28d90309ca2`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:41.813221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-784bc7b6-9592-4506-a10a-171ed9207104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:41.859381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:41.894038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 954/1938] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:41.907412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bc7b6-9592-4506-a10a-171ed9207104 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:41.908672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-784bc7b6-9592-4506-a10a-171ed9207104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:41.921018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785c3ab021df4c02bc24ff7c6d07a319', 'request_id': 'req-83a0908e-b416-4324-860b-726f9861dc67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:41.927248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785c3ab021df4c02bc24ff7c6d07a319', 'request_id': 'req-83a0908e-b416-4324-860b-726f9861dc67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:41.966602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bc7b6-9592-4506-a10a-171ed9207104 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:42.013770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5dbc3-abd2-4c58-a484-6b9433b084c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:42.014406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5dbc3-abd2-4c58-a484-6b9433b084c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:42.014674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5dbc3-abd2-4c58-a484-6b9433b084c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:42.014972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5dbc3-abd2-4c58-a484-6b9433b084c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:42.038008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785c3ab021df4c02bc24ff7c6d07a319', 'request_id': 'req-83a0908e-b416-4324-860b-726f9861dc67'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.049091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] The token cache is being invalidate because project 785c3ab021df4c02bc24ff7c6d07a319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:42.049599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.050790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.053642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 985/1939] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:41 2022] DELETE /identity/v3/projects/785c3ab021df4c02bc24ff7c6d07a319 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:42.056339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:42.057632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:42.075704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5dbc3-abd2-4c58-a484-6b9433b084c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:42.077378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5dbc3-abd2-4c58-a484-6b9433b084c6 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:42.087779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 955/1940] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:43:42.126359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0908e-b416-4324-860b-726f9861dc67 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:42.131709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:42.132352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:42.133914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:42.133914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] PATH_INFO: `/v3/users/9c9fc09ee2f04634bf0fa259d46d7bc7` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:42.134081 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c9fc09ee2f04634bf0fa259d46d7bc7)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:42.135970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:42.153717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5dbc3-abd2-4c58-a484-6b9433b084c6 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:42.164100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5dbc3-abd2-4c58-a484-6b9433b084c6 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:42.168677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:42.169292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:42.169567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:42.169837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] PATH_INFO: `/v3/users/42b6c20564564ba185a3b84177c10a81` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:42.170986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b6c20564564ba185a3b84177c10a81)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:42.172913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:42.317560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:42.382648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c9fc09ee2f04634bf0fa259d46d7bc7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.394450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c9fc09ee2f04634bf0fa259d46d7bc7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.395124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c9fc09ee2f04634bf0fa259d46d7bc7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.408579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b6c20564564ba185a3b84177c10a81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.409217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b6c20564564ba185a3b84177c10a81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.421251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b6c20564564ba185a3b84177c10a81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.475965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:42.476579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c9fc09ee2f04634bf0fa259d46d7bc7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.477260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0f623c7c-ffa5-4af0-b570-b1e5b221566f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c9fc09ee2f04634bf0fa259d46d7bc7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.490524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 986/1941] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:42 2022] DELETE /identity/v3/users/9c9fc09ee2f04634bf0fa259d46d7bc7 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:42.543954 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b6c20564564ba185a3b84177c10a81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.544579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b6c20564564ba185a3b84177c10a81'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:42.559339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 956/1942] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:42 2022] DELETE /identity/v3/users/42b6c20564564ba185a3b84177c10a81 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:42.566368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff5550-7351-41d5-b4ce-3540ea9a79b7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:42.566546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff5550-7351-41d5-b4ce-3540ea9a79b7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:42.566828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff5550-7351-41d5-b4ce-3540ea9a79b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:42.588438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-57ff5550-7351-41d5-b4ce-3540ea9a79b7 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:42.589841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-57ff5550-7351-41d5-b4ce-3540ea9a79b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:42.653223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 987/1943] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:42.661532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:42.663402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:42.674417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:42.701903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:42.746879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b6b70-e02a-4b13-955a-a3ca188b717f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:42.750457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8fd97-03c4-424a-9959-0df880e78cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:42.751072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8fd97-03c4-424a-9959-0df880e78cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:42.751340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8fd97-03c4-424a-9959-0df880e78cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:42.751645 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8fd97-03c4-424a-9959-0df880e78cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:42.760829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8fd97-03c4-424a-9959-0df880e78cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:42.762309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8fd97-03c4-424a-9959-0df880e78cb8 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:42.773896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 957/1944] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:42.937813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff5550-7351-41d5-b4ce-3540ea9a79b7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:42.939825 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-57ff5550-7351-41d5-b4ce-3540ea9a79b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:42.955323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff5550-7351-41d5-b4ce-3540ea9a79b7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:42.967686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:42.968352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:42.968672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:42.969005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] PATH_INFO: `/v3/projects/19b2704415014d1e8ffb0c6e6dfa29a2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:42.970216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b2704415014d1e8ffb0c6e6dfa29a2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:42.972907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:43.082265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b2704415014d1e8ffb0c6e6dfa29a2', 'request_id': 'req-9705ff06-1d01-42bc-8882-95d77b14fc23'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.088386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b2704415014d1e8ffb0c6e6dfa29a2', 'request_id': 'req-9705ff06-1d01-42bc-8882-95d77b14fc23'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.206863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b2704415014d1e8ffb0c6e6dfa29a2', 'request_id': 'req-9705ff06-1d01-42bc-8882-95d77b14fc23'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.218491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] The token cache is being invalidate because project 19b2704415014d1e8ffb0c6e6dfa29a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:43.218787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.219427 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9705ff06-1d01-42bc-8882-95d77b14fc23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.221566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 988/1945] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:42 2022] DELETE /identity/v3/projects/19b2704415014d1e8ffb0c6e6dfa29a2 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:43.227491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d829859-3d0d-402f-822a-817b2b0bac4d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.227779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d829859-3d0d-402f-822a-817b2b0bac4d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.228081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d829859-3d0d-402f-822a-817b2b0bac4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.247935 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9d829859-3d0d-402f-822a-817b2b0bac4d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:43.249163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d829859-3d0d-402f-822a-817b2b0bac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:43.268711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:43.307953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 958/1946] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:43.317504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cde36-c279-4dd2-91fa-10ad3d70f594 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.317805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cde36-c279-4dd2-91fa-10ad3d70f594 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.318067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cde36-c279-4dd2-91fa-10ad3d70f594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.323313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d829859-3d0d-402f-822a-817b2b0bac4d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:43.324700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d829859-3d0d-402f-822a-817b2b0bac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:43.340178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d829859-3d0d-402f-822a-817b2b0bac4d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:43.341016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-291cde36-c279-4dd2-91fa-10ad3d70f594 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:43.342272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-291cde36-c279-4dd2-91fa-10ad3d70f594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:43.345386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:43.347659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.347951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.348236 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] PATH_INFO: `/v3/projects/dee0411306364b30b9be059c451fb538` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.349338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dee0411306364b30b9be059c451fb538)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:43.351294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 989/1947] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:43.351402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:43.357841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cde36-c279-4dd2-91fa-10ad3d70f594 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:43.367636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cde36-c279-4dd2-91fa-10ad3d70f594 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:43.372325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2c0f-2fd7-41b4-ad49-9bf4c292d5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:43.372962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2c0f-2fd7-41b4-ad49-9bf4c292d5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.373241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2c0f-2fd7-41b4-ad49-9bf4c292d5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.373635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2c0f-2fd7-41b4-ad49-9bf4c292d5c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.374350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2c0f-2fd7-41b4-ad49-9bf4c292d5c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:43.374656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2c0f-2fd7-41b4-ad49-9bf4c292d5c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:43.376242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2c0f-2fd7-41b4-ad49-9bf4c292d5c6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:43.388414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 990/1948] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:43:43.393462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2c0f-2fd7-41b4-ad49-9bf4c292d5c6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:43.402808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2c0f-2fd7-41b4-ad49-9bf4c292d5c6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:43.407300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b7b70-31d4-4400-b301-32a307b7aaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:43.407870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b7b70-31d4-4400-b301-32a307b7aaac admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.408142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b7b70-31d4-4400-b301-32a307b7aaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.408408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b7b70-31d4-4400-b301-32a307b7aaac admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.409015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b7b70-31d4-4400-b301-32a307b7aaac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:43.410921 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b7b70-31d4-4400-b301-32a307b7aaac admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:43.480557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dee0411306364b30b9be059c451fb538', 'request_id': 'req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.486803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dee0411306364b30b9be059c451fb538', 'request_id': 'req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.505620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 991/1949] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:43 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:43.509599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b7b70-31d4-4400-b301-32a307b7aaac admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:43.518515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:43.555211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dee0411306364b30b9be059c451fb538', 'request_id': 'req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.567181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] The token cache is being invalidate because project dee0411306364b30b9be059c451fb538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:43.567475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.568137 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.569702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 959/1950] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:43 2022] DELETE /identity/v3/projects/dee0411306364b30b9be059c451fb538 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:43.569832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b7b70-31d4-4400-b301-32a307b7aaac admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:43.573514 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:43.574733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914e71c-6468-4f4e-98f3-5d011cb09c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:43.574859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:43.575338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914e71c-6468-4f4e-98f3-5d011cb09c56 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.575620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914e71c-6468-4f4e-98f3-5d011cb09c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.575982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914e71c-6468-4f4e-98f3-5d011cb09c56 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.576545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914e71c-6468-4f4e-98f3-5d011cb09c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:43.578483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914e71c-6468-4f4e-98f3-5d011cb09c56 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:43.638264 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae589fe-f8f5-496e-9cb5-7e2ecafb8d6c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:43.643692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:43.644311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.644600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.644959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] PATH_INFO: `/v3/users/8338d61d164147f7b8f5f28d90309ca2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.646119 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8338d61d164147f7b8f5f28d90309ca2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:43.648054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:43.684030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 992/1951] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:43 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:43.687987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914e71c-6468-4f4e-98f3-5d011cb09c56 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:43.690033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2914e71c-6468-4f4e-98f3-5d011cb09c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:43.702085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914e71c-6468-4f4e-98f3-5d011cb09c56 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:43.707646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee64607-32eb-41ab-b9ca-c19b26779485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:43.708236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee64607-32eb-41ab-b9ca-c19b26779485 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.708509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee64607-32eb-41ab-b9ca-c19b26779485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.708795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee64607-32eb-41ab-b9ca-c19b26779485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.709419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee64607-32eb-41ab-b9ca-c19b26779485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:43.714805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee64607-32eb-41ab-b9ca-c19b26779485 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:43.721719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 993/1952] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:43.725521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee64607-32eb-41ab-b9ca-c19b26779485 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:43.734917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee64607-32eb-41ab-b9ca-c19b26779485 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:43.739601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28312c-8adc-42c8-a5a6-48f668b48a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:43.740188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28312c-8adc-42c8-a5a6-48f668b48a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.740466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28312c-8adc-42c8-a5a6-48f668b48a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.740742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28312c-8adc-42c8-a5a6-48f668b48a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.741356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28312c-8adc-42c8-a5a6-48f668b48a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:43.742929 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28312c-8adc-42c8-a5a6-48f668b48a39 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:43.753515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 994/1953] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:43.757355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28312c-8adc-42c8-a5a6-48f668b48a39 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:43.766695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28312c-8adc-42c8-a5a6-48f668b48a39 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:43.772979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da93a2-d1b7-49e4-9246-2a4172d9782f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:43.773598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da93a2-d1b7-49e4-9246-2a4172d9782f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.773871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da93a2-d1b7-49e4-9246-2a4172d9782f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.774161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da93a2-d1b7-49e4-9246-2a4172d9782f admin admin] PATH_INFO: `/v3/projects/ab5d0ffd6a7b4974a307edf38341fe8c/users/b792486a66694360bef064628b5a9b86/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.788327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da93a2-d1b7-49e4-9246-2a4172d9782f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab5d0ffd6a7b4974a307edf38341fe8c, user_id=b792486a66694360bef064628b5a9b86, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:43.791121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da93a2-d1b7-49e4-9246-2a4172d9782f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:43.866234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8338d61d164147f7b8f5f28d90309ca2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.866832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8338d61d164147f7b8f5f28d90309ca2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.879909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8338d61d164147f7b8f5f28d90309ca2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.882092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 995/1954] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:43 2022] PUT /identity/v3/projects/ab5d0ffd6a7b4974a307edf38341fe8c/users/b792486a66694360bef064628b5a9b86/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:43.893634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b6460-08b2-45cb-b891-570793631cd7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:43.894017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b6460-08b2-45cb-b891-570793631cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:43.894408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b6460-08b2-45cb-b891-570793631cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:43.922491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f64b6460-08b2-45cb-b891-570793631cd7 None None] MFA Rules not processed for user `b792486a66694360bef064628b5a9b86`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:43.924209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f64b6460-08b2-45cb-b891-570793631cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:43.983529 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8338d61d164147f7b8f5f28d90309ca2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.984246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8338d61d164147f7b8f5f28d90309ca2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:43.988427 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 996/1955] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:43.998803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 960/1956] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:43 2022] DELETE /identity/v3/users/8338d61d164147f7b8f5f28d90309ca2 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:44.000070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b6460-08b2-45cb-b891-570793631cd7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:44.001440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f64b6460-08b2-45cb-b891-570793631cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:44.042498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:44.068587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b6460-08b2-45cb-b891-570793631cd7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:44.120210 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd122b41-ae38-4e3c-8d8d-e6e3204fd75d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:44.120757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd122b41-ae38-4e3c-8d8d-e6e3204fd75d service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:44.121031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd122b41-ae38-4e3c-8d8d-e6e3204fd75d service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:44.121320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd122b41-ae38-4e3c-8d8d-e6e3204fd75d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:44.128372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:44.176935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd122b41-ae38-4e3c-8d8d-e6e3204fd75d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:44.178396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd122b41-ae38-4e3c-8d8d-e6e3204fd75d service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:44.187875 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 997/1957] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:44.206994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:44.208832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:44.277651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b53a9-07ef-4c2c-b83e-632c2cac6b5e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:44.283215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:44.283866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:44.284145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:44.284418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] PATH_INFO: `/v3/users/b792486a66694360bef064628b5a9b86` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:44.285551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b792486a66694360bef064628b5a9b86)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:44.287526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:44.433806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd122b41-ae38-4e3c-8d8d-e6e3204fd75d service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:44.434264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cd122b41-ae38-4e3c-8d8d-e6e3204fd75d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:44.448359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd122b41-ae38-4e3c-8d8d-e6e3204fd75d service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:44.452836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:44.453471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:44.453777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:44.454049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] PATH_INFO: `/v3/projects/c29a5549e70f436097e2aba9e62dbf5c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:44.455252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29a5549e70f436097e2aba9e62dbf5c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:44.457640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:44.508623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b792486a66694360bef064628b5a9b86'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:44.509382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b792486a66694360bef064628b5a9b86'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:44.521956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b792486a66694360bef064628b5a9b86'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:44.563551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29a5549e70f436097e2aba9e62dbf5c', 'request_id': 'req-2fa21a6c-61c9-4802-9cd9-cf15968811b8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:44.569886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29a5549e70f436097e2aba9e62dbf5c', 'request_id': 'req-2fa21a6c-61c9-4802-9cd9-cf15968811b8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:44.631775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b792486a66694360bef064628b5a9b86'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:44.632407 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d164207c-0549-48fc-b36e-95603ac60e52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b792486a66694360bef064628b5a9b86'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:44.645916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 961/1958] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:44 2022] DELETE /identity/v3/users/b792486a66694360bef064628b5a9b86 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:44.650216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70587d03-72be-408e-8735-e30cd7cd282b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:44.650502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70587d03-72be-408e-8735-e30cd7cd282b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:44.650772 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70587d03-72be-408e-8735-e30cd7cd282b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:44.672233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-70587d03-72be-408e-8735-e30cd7cd282b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:44.673431 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-70587d03-72be-408e-8735-e30cd7cd282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:44.721626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:44.737680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29a5549e70f436097e2aba9e62dbf5c', 'request_id': 'req-2fa21a6c-61c9-4802-9cd9-cf15968811b8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:44.738348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 962/1959] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:44.748253 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70587d03-72be-408e-8735-e30cd7cd282b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:44.748696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] The token cache is being invalidate because project c29a5549e70f436097e2aba9e62dbf5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:44.748982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:44.749620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2fa21a6c-61c9-4802-9cd9-cf15968811b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:44.749741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-70587d03-72be-408e-8735-e30cd7cd282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:44.751199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 998/1960] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:44 2022] DELETE /identity/v3/projects/c29a5549e70f436097e2aba9e62dbf5c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:44.755615 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c87d6-d7f8-4803-86d4-0665d87f26e8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:44.755905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c87d6-d7f8-4803-86d4-0665d87f26e8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:44.756231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c87d6-d7f8-4803-86d4-0665d87f26e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:44.775958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ca1c87d6-d7f8-4803-86d4-0665d87f26e8 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:44.777153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ca1c87d6-d7f8-4803-86d4-0665d87f26e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:44.785559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 999/1961] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:44.793276 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d998ed-0358-43c4-a870-0a9ffd4590c7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:44.793554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d998ed-0358-43c4-a870-0a9ffd4590c7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:44.793823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d998ed-0358-43c4-a870-0a9ffd4590c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:44.811512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70587d03-72be-408e-8735-e30cd7cd282b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:44.814122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-87d998ed-0358-43c4-a870-0a9ffd4590c7 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:44.815423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-87d998ed-0358-43c4-a870-0a9ffd4590c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:44.818510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6102c81-1b18-46cd-b105-083d884b25fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:44.818510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6102c81-1b18-46cd-b105-083d884b25fd service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:44.818510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6102c81-1b18-46cd-b105-083d884b25fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:44.818510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6102c81-1b18-46cd-b105-083d884b25fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:44.818874 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f6102c81-1b18-46cd-b105-083d884b25fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:44.824309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1000/1962] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:44.828999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d998ed-0358-43c4-a870-0a9ffd4590c7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:44.831441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6102c81-1b18-46cd-b105-083d884b25fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:44.832952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6102c81-1b18-46cd-b105-083d884b25fd service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:44.838380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d998ed-0358-43c4-a870-0a9ffd4590c7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:44.843126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 963/1963] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:43:44.844549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a27d61-9f90-41c1-a0d7-880a71f7fc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:44.845150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a27d61-9f90-41c1-a0d7-880a71f7fc05 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:44.846003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a27d61-9f90-41c1-a0d7-880a71f7fc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:44.846374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a27d61-9f90-41c1-a0d7-880a71f7fc05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:44.847022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a27d61-9f90-41c1-a0d7-880a71f7fc05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:44.847321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a27d61-9f90-41c1-a0d7-880a71f7fc05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:44.848918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a27d61-9f90-41c1-a0d7-880a71f7fc05 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:44.861182 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1001/1964] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:43:44.865715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6102c81-1b18-46cd-b105-083d884b25fd service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:44.875344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6102c81-1b18-46cd-b105-083d884b25fd service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:44.881269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1688-301a-4374-9988-74be9a8bdaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:44.881865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1688-301a-4374-9988-74be9a8bdaa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:44.882157 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1688-301a-4374-9988-74be9a8bdaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:44.882428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1688-301a-4374-9988-74be9a8bdaa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:44.883045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca1688-301a-4374-9988-74be9a8bdaa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:44.884998 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca1688-301a-4374-9988-74be9a8bdaa0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:44.982861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 964/1965] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:44 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:44.987592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a27d61-9f90-41c1-a0d7-880a71f7fc05 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:44.997380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a27d61-9f90-41c1-a0d7-880a71f7fc05 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.000814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:45.061459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3999e-8baa-42e3-8629-21ad43118ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.063044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3999e-8baa-42e3-8629-21ad43118ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.063321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3999e-8baa-42e3-8629-21ad43118ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.063592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3999e-8baa-42e3-8629-21ad43118ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.064360 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3999e-8baa-42e3-8629-21ad43118ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.066324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3999e-8baa-42e3-8629-21ad43118ad9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.169557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1002/1966] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:44 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:45.174430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1688-301a-4374-9988-74be9a8bdaa0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.184750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1688-301a-4374-9988-74be9a8bdaa0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.189600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d3118-1f4a-4b6e-a947-d211a832ac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.190155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d3118-1f4a-4b6e-a947-d211a832ac52 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.190440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d3118-1f4a-4b6e-a947-d211a832ac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.190706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d3118-1f4a-4b6e-a947-d211a832ac52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.191330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d3118-1f4a-4b6e-a947-d211a832ac52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.192893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d3118-1f4a-4b6e-a947-d211a832ac52 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.203263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 965/1967] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:45.207162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3999e-8baa-42e3-8629-21ad43118ad9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.217374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3999e-8baa-42e3-8629-21ad43118ad9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.222875 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f19bf9-1364-47b0-ad33-88ee5659132c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.223466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f19bf9-1364-47b0-ad33-88ee5659132c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.223739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f19bf9-1364-47b0-ad33-88ee5659132c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.226190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f19bf9-1364-47b0-ad33-88ee5659132c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.226190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f19bf9-1364-47b0-ad33-88ee5659132c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.226481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f19bf9-1364-47b0-ad33-88ee5659132c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.237501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1003/1968] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:45.241864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d3118-1f4a-4b6e-a947-d211a832ac52 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.249118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f19bf9-1364-47b0-ad33-88ee5659132c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.249118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-87f19bf9-1364-47b0-ad33-88ee5659132c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:45.258593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d3118-1f4a-4b6e-a947-d211a832ac52 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.263847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38450da-fbe6-4311-bc5b-51e7718008cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.264570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38450da-fbe6-4311-bc5b-51e7718008cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.264843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38450da-fbe6-4311-bc5b-51e7718008cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.266369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38450da-fbe6-4311-bc5b-51e7718008cf admin admin] PATH_INFO: `/v3/projects/166224e84d534e25ab1ca7df3380034a/users/e17e82fab7c245d3ad003dead541839d/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.266479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f19bf9-1364-47b0-ad33-88ee5659132c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.273347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.274275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.274788 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.275640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] PATH_INFO: `/v3/projects/ab5d0ffd6a7b4974a307edf38341fe8c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.279381 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab5d0ffd6a7b4974a307edf38341fe8c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.282293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.284294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38450da-fbe6-4311-bc5b-51e7718008cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=166224e84d534e25ab1ca7df3380034a, user_id=e17e82fab7c245d3ad003dead541839d, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.287963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38450da-fbe6-4311-bc5b-51e7718008cf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.335675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 966/1969] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:45 2022] PUT /identity/v3/projects/166224e84d534e25ab1ca7df3380034a/users/e17e82fab7c245d3ad003dead541839d/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:45.346505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51133cfe-f40d-4fa3-a6ab-1eda40e43864 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.346790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51133cfe-f40d-4fa3-a6ab-1eda40e43864 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.347058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51133cfe-f40d-4fa3-a6ab-1eda40e43864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.366489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab5d0ffd6a7b4974a307edf38341fe8c', 'request_id': 'req-3b60ae76-b2ca-48fe-88d8-738150f77b0c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:45.372839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab5d0ffd6a7b4974a307edf38341fe8c', 'request_id': 'req-3b60ae76-b2ca-48fe-88d8-738150f77b0c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:45.379427 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-51133cfe-f40d-4fa3-a6ab-1eda40e43864 None None] MFA Rules not processed for user `e17e82fab7c245d3ad003dead541839d`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:45.380780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-51133cfe-f40d-4fa3-a6ab-1eda40e43864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:45.438360 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab5d0ffd6a7b4974a307edf38341fe8c', 'request_id': 'req-3b60ae76-b2ca-48fe-88d8-738150f77b0c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:45.449927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 967/1970] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:45.449927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51133cfe-f40d-4fa3-a6ab-1eda40e43864 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.451462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] The token cache is being invalidate because project ab5d0ffd6a7b4974a307edf38341fe8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:45.451759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:45.452424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b60ae76-b2ca-48fe-88d8-738150f77b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:45.454105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1004/1971] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:45 2022] DELETE /identity/v3/projects/ab5d0ffd6a7b4974a307edf38341fe8c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:45.459467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c979e-fc9c-4690-a5b4-eab9117671aa None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.459749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c979e-fc9c-4690-a5b4-eab9117671aa None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.460048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c979e-fc9c-4690-a5b4-eab9117671aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.476536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:45.481948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7c0c979e-fc9c-4690-a5b4-eab9117671aa None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:45.483274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7c0c979e-fc9c-4690-a5b4-eab9117671aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:45.518224 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51133cfe-f40d-4fa3-a6ab-1eda40e43864 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.523060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfa347-69d2-4746-b49d-18bb8399dd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.523673 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfa347-69d2-4746-b49d-18bb8399dd72 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.524121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfa347-69d2-4746-b49d-18bb8399dd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.524254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfa347-69d2-4746-b49d-18bb8399dd72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.524880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfa347-69d2-4746-b49d-18bb8399dd72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.526813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfa347-69d2-4746-b49d-18bb8399dd72 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.543047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1005/1972] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:45.550751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da2286-9609-4bd6-9f63-7663a0a8674e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.551066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da2286-9609-4bd6-9f63-7663a0a8674e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.551458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da2286-9609-4bd6-9f63-7663a0a8674e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.570418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-21da2286-9609-4bd6-9f63-7663a0a8674e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:45.571604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-21da2286-9609-4bd6-9f63-7663a0a8674e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:45.579223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1006/1973] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:45.583831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da2286-9609-4bd6-9f63-7663a0a8674e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.588137 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:45.588813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 968/1974] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:45 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:45.597466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da2286-9609-4bd6-9f63-7663a0a8674e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.600459 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfa347-69d2-4746-b49d-18bb8399dd72 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.602365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-41dfa347-69d2-4746-b49d-18bb8399dd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:45.651265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef947e3c-dc0c-4df3-95e6-9d3b3f41b343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.651888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef947e3c-dc0c-4df3-95e6-9d3b3f41b343 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.652209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef947e3c-dc0c-4df3-95e6-9d3b3f41b343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.652491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef947e3c-dc0c-4df3-95e6-9d3b3f41b343 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.653135 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef947e3c-dc0c-4df3-95e6-9d3b3f41b343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:45.653442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef947e3c-dc0c-4df3-95e6-9d3b3f41b343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.655204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef947e3c-dc0c-4df3-95e6-9d3b3f41b343 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.670570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1007/1975] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:43:45.670737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfa347-69d2-4746-b49d-18bb8399dd72 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.671933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef947e3c-dc0c-4df3-95e6-9d3b3f41b343 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.672403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae14600-db36-46bc-8701-85bf0e846f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.673014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae14600-db36-46bc-8701-85bf0e846f33 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.673409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae14600-db36-46bc-8701-85bf0e846f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.673686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae14600-db36-46bc-8701-85bf0e846f33 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.674350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae14600-db36-46bc-8701-85bf0e846f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.676295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae14600-db36-46bc-8701-85bf0e846f33 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.681499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef947e3c-dc0c-4df3-95e6-9d3b3f41b343 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.686121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38782866-d9c1-4265-9b2d-622c0aed8431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.686743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38782866-d9c1-4265-9b2d-622c0aed8431 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.687016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38782866-d9c1-4265-9b2d-622c0aed8431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.687539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38782866-d9c1-4265-9b2d-622c0aed8431 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.688191 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38782866-d9c1-4265-9b2d-622c0aed8431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.690663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38782866-d9c1-4265-9b2d-622c0aed8431 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.778925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 969/1976] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:45 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:45.779781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1008/1977] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:45 2022] POST /identity/v3/projects => generated 361 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:45.782656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae14600-db36-46bc-8701-85bf0e846f33 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.783694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38782866-d9c1-4265-9b2d-622c0aed8431 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.793149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae14600-db36-46bc-8701-85bf0e846f33 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.850077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38782866-d9c1-4265-9b2d-622c0aed8431 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.854905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d31217-e56e-453e-afa6-637fddc94da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.855779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb183e-7072-4926-a0e6-7a0d9f99f293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.855931 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d31217-e56e-453e-afa6-637fddc94da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.856052 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d31217-e56e-453e-afa6-637fddc94da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.856308 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d31217-e56e-453e-afa6-637fddc94da8 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.856398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb183e-7072-4926-a0e6-7a0d9f99f293 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.856630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb183e-7072-4926-a0e6-7a0d9f99f293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.856903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb183e-7072-4926-a0e6-7a0d9f99f293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.856992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d31217-e56e-453e-afa6-637fddc94da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.857576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb183e-7072-4926-a0e6-7a0d9f99f293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.858876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d31217-e56e-453e-afa6-637fddc94da8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.859199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb183e-7072-4926-a0e6-7a0d9f99f293 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.868978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 970/1978] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:45.872831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb183e-7072-4926-a0e6-7a0d9f99f293 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.883117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb183e-7072-4926-a0e6-7a0d9f99f293 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.887759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e1b5f-5430-41be-8ed3-bc3966091928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.888352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e1b5f-5430-41be-8ed3-bc3966091928 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.888629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e1b5f-5430-41be-8ed3-bc3966091928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.888899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e1b5f-5430-41be-8ed3-bc3966091928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.889528 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e1b5f-5430-41be-8ed3-bc3966091928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.891199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e1b5f-5430-41be-8ed3-bc3966091928 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.901895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 971/1979] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:45.906056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e1b5f-5430-41be-8ed3-bc3966091928 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.915639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e1b5f-5430-41be-8ed3-bc3966091928 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.920255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc08b01-0175-40cc-9ff4-207f01d9d02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.920877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc08b01-0175-40cc-9ff4-207f01d9d02d admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.921174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc08b01-0175-40cc-9ff4-207f01d9d02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.921457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc08b01-0175-40cc-9ff4-207f01d9d02d admin admin] PATH_INFO: `/v3/projects/d84f33237945468cb3c659eb9e5025e2/users/5851db4f7c6c4990baae7c9cb27b27b5/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.936939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc08b01-0175-40cc-9ff4-207f01d9d02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d84f33237945468cb3c659eb9e5025e2, user_id=5851db4f7c6c4990baae7c9cb27b27b5, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.940409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc08b01-0175-40cc-9ff4-207f01d9d02d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.950008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1009/1980] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:45 2022] POST /identity/v3/users => generated 281 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:45.953970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d31217-e56e-453e-afa6-637fddc94da8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.964137 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d31217-e56e-453e-afa6-637fddc94da8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:45.968885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038002f-2606-4fcc-991a-8126ac8d51d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:45.969398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038002f-2606-4fcc-991a-8126ac8d51d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:45.969689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038002f-2606-4fcc-991a-8126ac8d51d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:45.969943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038002f-2606-4fcc-991a-8126ac8d51d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:45.970628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038002f-2606-4fcc-991a-8126ac8d51d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:45.972185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038002f-2606-4fcc-991a-8126ac8d51d8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:45.982015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1010/1981] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:45.985724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038002f-2606-4fcc-991a-8126ac8d51d8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:45.995403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038002f-2606-4fcc-991a-8126ac8d51d8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.000095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4f209-497d-4cf6-ac2a-bb124349585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.000756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4f209-497d-4cf6-ac2a-bb124349585d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.000999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4f209-497d-4cf6-ac2a-bb124349585d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.001268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4f209-497d-4cf6-ac2a-bb124349585d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.002502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4f209-497d-4cf6-ac2a-bb124349585d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.004141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4f209-497d-4cf6-ac2a-bb124349585d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.014612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1011/1982] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:46.019211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4f209-497d-4cf6-ac2a-bb124349585d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.028414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4f209-497d-4cf6-ac2a-bb124349585d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.030438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:46.036239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 972/1983] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:45 2022] PUT /identity/v3/projects/d84f33237945468cb3c659eb9e5025e2/users/5851db4f7c6c4990baae7c9cb27b27b5/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:46.047616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570491e-eabe-4ab0-a33e-512d1b68cf45 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.047946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570491e-eabe-4ab0-a33e-512d1b68cf45 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.048216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570491e-eabe-4ab0-a33e-512d1b68cf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.069350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:46.076882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b570491e-eabe-4ab0-a33e-512d1b68cf45 None None] MFA Rules not processed for user `5851db4f7c6c4990baae7c9cb27b27b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:46.078101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b570491e-eabe-4ab0-a33e-512d1b68cf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:46.085315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85d0e-cc43-49a2-9231-10293b018281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.085981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85d0e-cc43-49a2-9231-10293b018281 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.086276 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85d0e-cc43-49a2-9231-10293b018281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.086546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85d0e-cc43-49a2-9231-10293b018281 admin admin] PATH_INFO: `/v3/projects/7d06609ff3244630861ab6c3310add2e/users/4d3f31fe7fe0485e9d846972c3f6632f/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.101830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f85d0e-cc43-49a2-9231-10293b018281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d06609ff3244630861ab6c3310add2e, user_id=4d3f31fe7fe0485e9d846972c3f6632f, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.104699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f85d0e-cc43-49a2-9231-10293b018281 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.143049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 973/1984] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:46.148095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570491e-eabe-4ab0-a33e-512d1b68cf45 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.159217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570491e-eabe-4ab0-a33e-512d1b68cf45 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.159367 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1012/1985] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:46 2022] PUT /identity/v3/projects/7d06609ff3244630861ab6c3310add2e/users/4d3f31fe7fe0485e9d846972c3f6632f/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:46.160192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85d0e-cc43-49a2-9231-10293b018281 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.162302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a4f85d0e-cc43-49a2-9231-10293b018281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:46.217107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae0e9c-bc94-4a26-99f3-2a487feb6aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.217815 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae0e9c-bc94-4a26-99f3-2a487feb6aaa admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.218187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae0e9c-bc94-4a26-99f3-2a487feb6aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.218531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae0e9c-bc94-4a26-99f3-2a487feb6aaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.225068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae0e9c-bc94-4a26-99f3-2a487feb6aaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.225068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae0e9c-bc94-4a26-99f3-2a487feb6aaa admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.225796 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85d0e-cc43-49a2-9231-10293b018281 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.230498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.231165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.231445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.231743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] PATH_INFO: `/v3/users/75b743d09e9443feb75f51aa9c378a8b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.232866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75b743d09e9443feb75f51aa9c378a8b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.234800 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.290321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 974/1986] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:46 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:46.291741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e3b42-ebb8-4301-b7b4-d23e4ce0d6c5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.292030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e3b42-ebb8-4301-b7b4-d23e4ce0d6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.292307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e3b42-ebb8-4301-b7b4-d23e4ce0d6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.321115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f21e3b42-ebb8-4301-b7b4-d23e4ce0d6c5 None None] MFA Rules not processed for user `4d3f31fe7fe0485e9d846972c3f6632f`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:46.322380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f21e3b42-ebb8-4301-b7b4-d23e4ce0d6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:46.371799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75b743d09e9443feb75f51aa9c378a8b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:46.385402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75b743d09e9443feb75f51aa9c378a8b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:46.386159 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75b743d09e9443feb75f51aa9c378a8b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:46.387658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 975/1987] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:46.392130 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e3b42-ebb8-4301-b7b4-d23e4ce0d6c5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.392536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:46.443760 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75b743d09e9443feb75f51aa9c378a8b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:46.444432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75b743d09e9443feb75f51aa9c378a8b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:46.451080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e3b42-ebb8-4301-b7b4-d23e4ce0d6c5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.455837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3b88f-48e6-4cf0-b329-cb9aa561d906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.456464 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3b88f-48e6-4cf0-b329-cb9aa561d906 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.456747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3b88f-48e6-4cf0-b329-cb9aa561d906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.457053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3b88f-48e6-4cf0-b329-cb9aa561d906 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.457275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1013/1988] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:46 2022] DELETE /identity/v3/users/75b743d09e9443feb75f51aa9c378a8b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:46.460237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3b88f-48e6-4cf0-b329-cb9aa561d906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.460337 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.460337 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:46.460496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3b88f-48e6-4cf0-b329-cb9aa561d906 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.505573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:46.515679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d236e-ef10-4764-b233-7a196aee8808 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.519567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3dcd1-094e-4db2-96ec-a24108fbd776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.520178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3dcd1-094e-4db2-96ec-a24108fbd776 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.520458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3dcd1-094e-4db2-96ec-a24108fbd776 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.520731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3dcd1-094e-4db2-96ec-a24108fbd776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.522489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a1d3dcd1-094e-4db2-96ec-a24108fbd776 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:46.538704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 976/1989] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:46 2022] POST /identity/v3/users => generated 300 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:46.542113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071b19d-4be4-4106-be88-433232e25a26 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.542113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071b19d-4be4-4106-be88-433232e25a26 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.542113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071b19d-4be4-4106-be88-433232e25a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.560591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c071b19d-4be4-4106-be88-433232e25a26 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:46.561815 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c071b19d-4be4-4106-be88-433232e25a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:46.582891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3dcd1-094e-4db2-96ec-a24108fbd776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.584317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3dcd1-094e-4db2-96ec-a24108fbd776 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.593938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1014/1990] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:46.594777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3dcd1-094e-4db2-96ec-a24108fbd776 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.596596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a1d3dcd1-094e-4db2-96ec-a24108fbd776 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:46.628341 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 977/1991] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:46.629384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071b19d-4be4-4106-be88-433232e25a26 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.630835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c071b19d-4be4-4106-be88-433232e25a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:46.642223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071b19d-4be4-4106-be88-433232e25a26 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.647567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.648219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.648497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.648765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] PATH_INFO: `/v3/users/4d3f31fe7fe0485e9d846972c3f6632f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.649882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d3f31fe7fe0485e9d846972c3f6632f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.651849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.663008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3dcd1-094e-4db2-96ec-a24108fbd776 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.667688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fe274-cbd0-4fa1-a2fe-c42d801a98e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.668284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fe274-cbd0-4fa1-a2fe-c42d801a98e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.668556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fe274-cbd0-4fa1-a2fe-c42d801a98e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.668830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fe274-cbd0-4fa1-a2fe-c42d801a98e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.669489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fe274-cbd0-4fa1-a2fe-c42d801a98e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.671094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fe274-cbd0-4fa1-a2fe-c42d801a98e7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.681453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1015/1992] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:46.682422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fe274-cbd0-4fa1-a2fe-c42d801a98e7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.684062 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d15fe274-cbd0-4fa1-a2fe-c42d801a98e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:46.738862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fe274-cbd0-4fa1-a2fe-c42d801a98e7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.742437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008c2ed-bbb2-465f-8b26-6bdcdedcc3d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.743072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c2ed-bbb2-465f-8b26-6bdcdedcc3d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.743348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c2ed-bbb2-465f-8b26-6bdcdedcc3d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.743613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008c2ed-bbb2-465f-8b26-6bdcdedcc3d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.752291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008c2ed-bbb2-465f-8b26-6bdcdedcc3d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.753692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008c2ed-bbb2-465f-8b26-6bdcdedcc3d7 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.763029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1016/1993] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:46.763825 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008c2ed-bbb2-465f-8b26-6bdcdedcc3d7 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.773333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008c2ed-bbb2-465f-8b26-6bdcdedcc3d7 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.778366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a3bf1-d1ea-4c01-8fa3-76571504a9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.779014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a3bf1-d1ea-4c01-8fa3-76571504a9de admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.779319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a3bf1-d1ea-4c01-8fa3-76571504a9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.779556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a3bf1-d1ea-4c01-8fa3-76571504a9de admin admin] PATH_INFO: `/v3/projects/2c83dd71632b4ea09876f14a39ff6132/users/77dd0152b25b4e77864185b385671d72/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.796048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a3bf1-d1ea-4c01-8fa3-76571504a9de admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c83dd71632b4ea09876f14a39ff6132, user_id=77dd0152b25b4e77864185b385671d72, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.798818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a3bf1-d1ea-4c01-8fa3-76571504a9de admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.856242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d3f31fe7fe0485e9d846972c3f6632f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:46.857310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d3f31fe7fe0485e9d846972c3f6632f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:46.877683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d3f31fe7fe0485e9d846972c3f6632f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:46.889767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1017/1994] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:46 2022] PUT /identity/v3/projects/2c83dd71632b4ea09876f14a39ff6132/users/77dd0152b25b4e77864185b385671d72/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:46.894442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a3bf1-d1ea-4c01-8fa3-76571504a9de admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.896071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1b2a3bf1-d1ea-4c01-8fa3-76571504a9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:46.911072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:46.958679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a3bf1-d1ea-4c01-8fa3-76571504a9de admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.963305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c8d0a-6210-4c0f-86c9-47ccb117f9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.963918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c8d0a-6210-4c0f-86c9-47ccb117f9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.964182 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c8d0a-6210-4c0f-86c9-47ccb117f9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.964457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c8d0a-6210-4c0f-86c9-47ccb117f9e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.965086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c8d0a-6210-4c0f-86c9-47ccb117f9e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.966717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c8d0a-6210-4c0f-86c9-47ccb117f9e2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:46.976723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1018/1995] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:46.980713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c8d0a-6210-4c0f-86c9-47ccb117f9e2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:46.983541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d3f31fe7fe0485e9d846972c3f6632f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:46.984223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7219f6c0-4609-4385-80f8-4f764a2d5514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d3f31fe7fe0485e9d846972c3f6632f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:46.990556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c8d0a-6210-4c0f-86c9-47ccb117f9e2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:46.995340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115af0c-dba8-4295-8293-358bf9585ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:46.995992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115af0c-dba8-4295-8293-358bf9585ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:46.996272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115af0c-dba8-4295-8293-358bf9585ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:46.996541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115af0c-dba8-4295-8293-358bf9585ba5 admin admin] PATH_INFO: `/v3/projects/2c83dd71632b4ea09876f14a39ff6132/users/77dd0152b25b4e77864185b385671d72/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:46.998475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115af0c-dba8-4295-8293-358bf9585ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c83dd71632b4ea09876f14a39ff6132, user_id=77dd0152b25b4e77864185b385671d72, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:46.999230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 978/1996] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:46 2022] DELETE /identity/v3/users/4d3f31fe7fe0485e9d846972c3f6632f => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:47.003241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666a95d-7737-462f-a8e1-ffbd04dc46e8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.003840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666a95d-7737-462f-a8e1-ffbd04dc46e8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.004029 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666a95d-7737-462f-a8e1-ffbd04dc46e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.004135 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115af0c-dba8-4295-8293-358bf9585ba5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:47.013893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:47.019701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1019/1997] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:46 2022] PUT /identity/v3/projects/2c83dd71632b4ea09876f14a39ff6132/users/77dd0152b25b4e77864185b385671d72/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:47.043171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213cdd-e85b-42f0-801d-70e0552dd88e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.043536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213cdd-e85b-42f0-801d-70e0552dd88e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.044267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213cdd-e85b-42f0-801d-70e0552dd88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.052932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0666a95d-7737-462f-a8e1-ffbd04dc46e8 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:47.054556 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0666a95d-7737-462f-a8e1-ffbd04dc46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:47.086164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7f213cdd-e85b-42f0-801d-70e0552dd88e None None] MFA Rules not processed for user `77dd0152b25b4e77864185b385671d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:47.087556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7f213cdd-e85b-42f0-801d-70e0552dd88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:47.124138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 979/1998] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:47.139037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666a95d-7737-462f-a8e1-ffbd04dc46e8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:47.141014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0666a95d-7737-462f-a8e1-ffbd04dc46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:47.167989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1020/1999] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:47.180006 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f213cdd-e85b-42f0-801d-70e0552dd88e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:47.181435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7f213cdd-e85b-42f0-801d-70e0552dd88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:47.203395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666a95d-7737-462f-a8e1-ffbd04dc46e8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:47.207619 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db001c-45b1-4b7e-807f-4e3f865574f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:47.208224 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db001c-45b1-4b7e-807f-4e3f865574f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.208498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db001c-45b1-4b7e-807f-4e3f865574f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.208765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db001c-45b1-4b7e-807f-4e3f865574f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.217737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db001c-45b1-4b7e-807f-4e3f865574f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:47.219180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db001c-45b1-4b7e-807f-4e3f865574f9 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:47.228757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 980/2000] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:43:47.240988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f213cdd-e85b-42f0-801d-70e0552dd88e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:47.244729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9204ab-3bae-46cf-89d9-864607f8a9b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:47.245350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9204ab-3bae-46cf-89d9-864607f8a9b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.245629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9204ab-3bae-46cf-89d9-864607f8a9b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.245948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9204ab-3bae-46cf-89d9-864607f8a9b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.255522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db001c-45b1-4b7e-807f-4e3f865574f9 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:47.257039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-47db001c-45b1-4b7e-807f-4e3f865574f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:47.258612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9204ab-3bae-46cf-89d9-864607f8a9b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:47.260402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9204ab-3bae-46cf-89d9-864607f8a9b1 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:47.272758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db001c-45b1-4b7e-807f-4e3f865574f9 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:47.280210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:47.281168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.281690 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.282288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] PATH_INFO: `/v3/projects/24bb9c290f79414289887665babfc45b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.284118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24bb9c290f79414289887665babfc45b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:47.284536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1021/2001] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:43:47.286930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:47.406759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24bb9c290f79414289887665babfc45b', 'request_id': 'req-4cb89597-8d46-4495-ac78-7ba57592ba54'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:47.413028 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24bb9c290f79414289887665babfc45b', 'request_id': 'req-4cb89597-8d46-4495-ac78-7ba57592ba54'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:47.512128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24bb9c290f79414289887665babfc45b', 'request_id': 'req-4cb89597-8d46-4495-ac78-7ba57592ba54'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:47.524903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] The token cache is being invalidate because project 24bb9c290f79414289887665babfc45b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:47.525415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:47.526694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4cb89597-8d46-4495-ac78-7ba57592ba54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:47.529282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 981/2002] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:47 2022] DELETE /identity/v3/projects/24bb9c290f79414289887665babfc45b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:47.533977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f1071-0fe2-491f-b421-b3fc77a275ba None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.534464 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f1071-0fe2-491f-b421-b3fc77a275ba None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.534886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f1071-0fe2-491f-b421-b3fc77a275ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.572462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8c1f1071-0fe2-491f-b421-b3fc77a275ba None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:47.573692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8c1f1071-0fe2-491f-b421-b3fc77a275ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:47.592215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:47.632875 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1022/2003] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:47.641074 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afebd16-2465-47ad-944b-9e6a85111d00 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.641555 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afebd16-2465-47ad-944b-9e6a85111d00 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.641733 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afebd16-2465-47ad-944b-9e6a85111d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.662206 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9afebd16-2465-47ad-944b-9e6a85111d00 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:47.663493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9afebd16-2465-47ad-944b-9e6a85111d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:47.671506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 982/2004] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:47.680727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f1071-0fe2-491f-b421-b3fc77a275ba None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:47.696892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f1071-0fe2-491f-b421-b3fc77a275ba None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:47.698206 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afebd16-2465-47ad-944b-9e6a85111d00 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:47.699625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9afebd16-2465-47ad-944b-9e6a85111d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:47.701075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bb508-95d9-4d34-92e6-8b0d754a3912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:47.702278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb508-95d9-4d34-92e6-8b0d754a3912 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.702881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb508-95d9-4d34-92e6-8b0d754a3912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.703297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb508-95d9-4d34-92e6-8b0d754a3912 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.704139 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bb508-95d9-4d34-92e6-8b0d754a3912 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:47.704667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bb508-95d9-4d34-92e6-8b0d754a3912 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:47.706591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bb508-95d9-4d34-92e6-8b0d754a3912 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:47.719184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afebd16-2465-47ad-944b-9e6a85111d00 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:47.724184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1023/2005] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:43:47.727261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bb508-95d9-4d34-92e6-8b0d754a3912 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:47.727702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:47.729075 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.729359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.729641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] PATH_INFO: `/v3/projects/7d06609ff3244630861ab6c3310add2e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.730989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d06609ff3244630861ab6c3310add2e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:47.732986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:47.740053 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bb508-95d9-4d34-92e6-8b0d754a3912 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:47.747102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c1924-6233-4466-b431-fefe802db6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:47.752153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c1924-6233-4466-b431-fefe802db6d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.752153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c1924-6233-4466-b431-fefe802db6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.752153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c1924-6233-4466-b431-fefe802db6d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.752153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c1924-6233-4466-b431-fefe802db6d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:47.752153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c1924-6233-4466-b431-fefe802db6d3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:47.759997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:47.897490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d06609ff3244630861ab6c3310add2e', 'request_id': 'req-21cc4604-469f-491a-946c-68bad57510af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:47.905839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d06609ff3244630861ab6c3310add2e', 'request_id': 'req-21cc4604-469f-491a-946c-68bad57510af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:47.910760 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1024/2006] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:47 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:47.914923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c1924-6233-4466-b431-fefe802db6d3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:47.975507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c1924-6233-4466-b431-fefe802db6d3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:47.980144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24ec73-e84b-4f22-b48e-2e4980dd313a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:47.980742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24ec73-e84b-4f22-b48e-2e4980dd313a admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:47.981020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24ec73-e84b-4f22-b48e-2e4980dd313a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:47.981292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24ec73-e84b-4f22-b48e-2e4980dd313a admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:47.981911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24ec73-e84b-4f22-b48e-2e4980dd313a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:47.983872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24ec73-e84b-4f22-b48e-2e4980dd313a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:47.986707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d06609ff3244630861ab6c3310add2e', 'request_id': 'req-21cc4604-469f-491a-946c-68bad57510af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:47.998553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] The token cache is being invalidate because project 7d06609ff3244630861ab6c3310add2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:47.998835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:47.999508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-21cc4604-469f-491a-946c-68bad57510af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:48.001113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 983/2007] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:47 2022] DELETE /identity/v3/projects/7d06609ff3244630861ab6c3310add2e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:48.005794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894883-75d4-41be-93b6-e8bcc0fdc41c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.006081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894883-75d4-41be-93b6-e8bcc0fdc41c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.006389 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894883-75d4-41be-93b6-e8bcc0fdc41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.025577 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6c894883-75d4-41be-93b6-e8bcc0fdc41c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:48.026796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6c894883-75d4-41be-93b6-e8bcc0fdc41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:48.035081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 984/2008] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:48.042337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938d5c8-dfdf-4da8-a862-a9fefbfb4541 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.042623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938d5c8-dfdf-4da8-a862-a9fefbfb4541 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.042895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938d5c8-dfdf-4da8-a862-a9fefbfb4541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.062409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5938d5c8-dfdf-4da8-a862-a9fefbfb4541 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:48.063574 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5938d5c8-dfdf-4da8-a862-a9fefbfb4541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:48.071298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 985/2009] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:48.075347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938d5c8-dfdf-4da8-a862-a9fefbfb4541 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.085037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938d5c8-dfdf-4da8-a862-a9fefbfb4541 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.089803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653a623-5f00-4e98-95b8-eb1923319e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.090441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653a623-5f00-4e98-95b8-eb1923319e05 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.090731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653a623-5f00-4e98-95b8-eb1923319e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.090986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653a623-5f00-4e98-95b8-eb1923319e05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.091629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653a623-5f00-4e98-95b8-eb1923319e05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:48.092017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653a623-5f00-4e98-95b8-eb1923319e05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.093559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653a623-5f00-4e98-95b8-eb1923319e05 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.109557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 986/2010] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:43:48.109726 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1025/2011] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:47 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:48.110466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653a623-5f00-4e98-95b8-eb1923319e05 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.110669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24ec73-e84b-4f22-b48e-2e4980dd313a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.112764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b653a623-5f00-4e98-95b8-eb1923319e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:48.120792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24ec73-e84b-4f22-b48e-2e4980dd313a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.125033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653a623-5f00-4e98-95b8-eb1923319e05 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.125975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5414bc-656d-48e1-a2db-6cfdf990d854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.126632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5414bc-656d-48e1-a2db-6cfdf990d854 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.126911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5414bc-656d-48e1-a2db-6cfdf990d854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.133707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5414bc-656d-48e1-a2db-6cfdf990d854 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.133707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5414bc-656d-48e1-a2db-6cfdf990d854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.133919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ebef7-946f-498c-b091-1c88f2f15bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.134040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5414bc-656d-48e1-a2db-6cfdf990d854 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.134126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ebef7-946f-498c-b091-1c88f2f15bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.134126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ebef7-946f-498c-b091-1c88f2f15bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.134126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ebef7-946f-498c-b091-1c88f2f15bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.134126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ebef7-946f-498c-b091-1c88f2f15bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.134126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ebef7-946f-498c-b091-1c88f2f15bd0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.144307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 987/2012] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:48.148212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ebef7-946f-498c-b091-1c88f2f15bd0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.158668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ebef7-946f-498c-b091-1c88f2f15bd0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.163599 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a1139-6ba0-41dd-9df7-31ca52153471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.164189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a1139-6ba0-41dd-9df7-31ca52153471 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.164468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a1139-6ba0-41dd-9df7-31ca52153471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.164742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a1139-6ba0-41dd-9df7-31ca52153471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.165372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a1139-6ba0-41dd-9df7-31ca52153471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.166982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a1139-6ba0-41dd-9df7-31ca52153471 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.177814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 988/2013] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:48.181871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a1139-6ba0-41dd-9df7-31ca52153471 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.192183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a1139-6ba0-41dd-9df7-31ca52153471 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.196837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b43e41-4225-4feb-ad2b-1052148d6a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.197470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b43e41-4225-4feb-ad2b-1052148d6a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.201666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b43e41-4225-4feb-ad2b-1052148d6a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.201666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b43e41-4225-4feb-ad2b-1052148d6a0c admin admin] PATH_INFO: `/v3/projects/2d12a9aff428415f8618a620e135a41e/users/c3c988c59510440f861f88b9d6b1fb88/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.212484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b43e41-4225-4feb-ad2b-1052148d6a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d12a9aff428415f8618a620e135a41e, user_id=c3c988c59510440f861f88b9d6b1fb88, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.215191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b43e41-4225-4feb-ad2b-1052148d6a0c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.258951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1026/2014] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:48 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:48.263010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5414bc-656d-48e1-a2db-6cfdf990d854 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.288687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 989/2015] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:48 2022] PUT /identity/v3/projects/2d12a9aff428415f8618a620e135a41e/users/c3c988c59510440f861f88b9d6b1fb88/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:48.298836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddff6c1-cd9a-44b8-9248-6e635b7b7620 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.299142 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddff6c1-cd9a-44b8-9248-6e635b7b7620 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.299429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddff6c1-cd9a-44b8-9248-6e635b7b7620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.322342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:48.326263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5414bc-656d-48e1-a2db-6cfdf990d854 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.327032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dddff6c1-cd9a-44b8-9248-6e635b7b7620 None None] MFA Rules not processed for user `c3c988c59510440f861f88b9d6b1fb88`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:48.328257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dddff6c1-cd9a-44b8-9248-6e635b7b7620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:48.385050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39125784-bce6-4c05-a1a2-98af17221960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.385585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39125784-bce6-4c05-a1a2-98af17221960 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.385864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39125784-bce6-4c05-a1a2-98af17221960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.386133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39125784-bce6-4c05-a1a2-98af17221960 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.386806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39125784-bce6-4c05-a1a2-98af17221960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.388790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39125784-bce6-4c05-a1a2-98af17221960 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.393183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 990/2016] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:48.404887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddff6c1-cd9a-44b8-9248-6e635b7b7620 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.406309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dddff6c1-cd9a-44b8-9248-6e635b7b7620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:48.434973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:48.464921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddff6c1-cd9a-44b8-9248-6e635b7b7620 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.468508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7869f48-0c15-4e6e-8fce-2edf2a8582f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.469115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7869f48-0c15-4e6e-8fce-2edf2a8582f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.469395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7869f48-0c15-4e6e-8fce-2edf2a8582f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.469668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7869f48-0c15-4e6e-8fce-2edf2a8582f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.478955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7869f48-0c15-4e6e-8fce-2edf2a8582f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.480377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7869f48-0c15-4e6e-8fce-2edf2a8582f0 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.483848 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1027/2017] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:48 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:48.487622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39125784-bce6-4c05-a1a2-98af17221960 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.493083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 991/2018] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:48.501850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39125784-bce6-4c05-a1a2-98af17221960 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.505090 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2130f-ccfe-4ad8-aa0a-36bd70e15229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.505721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2130f-ccfe-4ad8-aa0a-36bd70e15229 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.506363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2130f-ccfe-4ad8-aa0a-36bd70e15229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.506641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2130f-ccfe-4ad8-aa0a-36bd70e15229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.507288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2130f-ccfe-4ad8-aa0a-36bd70e15229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.508915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2130f-ccfe-4ad8-aa0a-36bd70e15229 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.523746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1028/2019] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:43:48.530225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7869f48-0c15-4e6e-8fce-2edf2a8582f0 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.531369 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2130f-ccfe-4ad8-aa0a-36bd70e15229 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.542692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7869f48-0c15-4e6e-8fce-2edf2a8582f0 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.544061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2130f-ccfe-4ad8-aa0a-36bd70e15229 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.548231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389d8a6-034d-46b2-9a3e-acef5950503d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.548843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389d8a6-034d-46b2-9a3e-acef5950503d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.549126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389d8a6-034d-46b2-9a3e-acef5950503d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.549404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389d8a6-034d-46b2-9a3e-acef5950503d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.550078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389d8a6-034d-46b2-9a3e-acef5950503d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.551063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.551702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389d8a6-034d-46b2-9a3e-acef5950503d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.552368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.552368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.552584 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] PATH_INFO: `/v3/users/c3c988c59510440f861f88b9d6b1fb88` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.554040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c988c59510440f861f88b9d6b1fb88)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.556003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.563251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 992/2020] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:48.567600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389d8a6-034d-46b2-9a3e-acef5950503d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.582961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389d8a6-034d-46b2-9a3e-acef5950503d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.587647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab4b53-af81-4896-be5c-cd52d005511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.588293 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab4b53-af81-4896-be5c-cd52d005511d admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.588576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab4b53-af81-4896-be5c-cd52d005511d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.588859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab4b53-af81-4896-be5c-cd52d005511d admin admin] PATH_INFO: `/v3/projects/e003d2bea5f04ac7a70fd2b7d2ee1f91/users/877575c9e0264241b9eab868a89cbbe8/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.604441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab4b53-af81-4896-be5c-cd52d005511d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e003d2bea5f04ac7a70fd2b7d2ee1f91, user_id=877575c9e0264241b9eab868a89cbbe8, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.607169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab4b53-af81-4896-be5c-cd52d005511d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.686446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 993/2021] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:48 2022] PUT /identity/v3/projects/e003d2bea5f04ac7a70fd2b7d2ee1f91/users/877575c9e0264241b9eab868a89cbbe8/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:48.687533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab4b53-af81-4896-be5c-cd52d005511d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.729498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c988c59510440f861f88b9d6b1fb88'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:48.742082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c988c59510440f861f88b9d6b1fb88'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:48.742837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c988c59510440f861f88b9d6b1fb88'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:48.744234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab4b53-af81-4896-be5c-cd52d005511d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:48.747930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185377e-d1b7-4ef7-be52-ccddb1543fab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:48.752559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185377e-d1b7-4ef7-be52-ccddb1543fab service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.752559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185377e-d1b7-4ef7-be52-ccddb1543fab service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.752559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185377e-d1b7-4ef7-be52-ccddb1543fab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.752559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1185377e-d1b7-4ef7-be52-ccddb1543fab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:48.812923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185377e-d1b7-4ef7-be52-ccddb1543fab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:48.814903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185377e-d1b7-4ef7-be52-ccddb1543fab service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:48.823637 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 994/2022] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:48.824976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d813ae7-437a-49b2-95b6-4fcc1e03d86c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.825256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d813ae7-437a-49b2-95b6-4fcc1e03d86c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.825525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d813ae7-437a-49b2-95b6-4fcc1e03d86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.844568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8d813ae7-437a-49b2-95b6-4fcc1e03d86c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:48.845817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8d813ae7-437a-49b2-95b6-4fcc1e03d86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:48.891005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:48.917091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c988c59510440f861f88b9d6b1fb88'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:48.918771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-18267f49-37ae-4458-a628-3ba99d33e668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c988c59510440f861f88b9d6b1fb88'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:48.925240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 995/2023] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:48.934650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d813ae7-437a-49b2-95b6-4fcc1e03d86c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:48.936601 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8d813ae7-437a-49b2-95b6-4fcc1e03d86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:48.943139 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1029/2024] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:43:48 2022] DELETE /identity/v3/users/c3c988c59510440f861f88b9d6b1fb88 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:43:48.948227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613bf4a-b607-4063-a0f1-5acc5cd8d240 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:48.948631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613bf4a-b607-4063-a0f1-5acc5cd8d240 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:48.949555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613bf4a-b607-4063-a0f1-5acc5cd8d240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:48.977953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2613bf4a-b607-4063-a0f1-5acc5cd8d240 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:48.979192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2613bf4a-b607-4063-a0f1-5acc5cd8d240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:49.003021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d813ae7-437a-49b2-95b6-4fcc1e03d86c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:49.029091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:49.040334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1030/2025] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:49.050697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613bf4a-b607-4063-a0f1-5acc5cd8d240 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:49.053820 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce6ca-239e-4286-956b-6f1a965ccb67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:49.054457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce6ca-239e-4286-956b-6f1a965ccb67 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:49.054734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce6ca-239e-4286-956b-6f1a965ccb67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:49.055004 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce6ca-239e-4286-956b-6f1a965ccb67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:49.060023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613bf4a-b607-4063-a0f1-5acc5cd8d240 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:49.064520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68204163-c85f-40d9-aef3-394856ca2458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:49.065111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68204163-c85f-40d9-aef3-394856ca2458 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:49.065390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68204163-c85f-40d9-aef3-394856ca2458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:49.065606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ce6ca-239e-4286-956b-6f1a965ccb67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:49.065705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68204163-c85f-40d9-aef3-394856ca2458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:49.067104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ce6ca-239e-4286-956b-6f1a965ccb67 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:49.077133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68204163-c85f-40d9-aef3-394856ca2458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:49.078049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 996/2026] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:49.087286 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68204163-c85f-40d9-aef3-394856ca2458 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:49.098805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1031/2027] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:49.557239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce6ca-239e-4286-956b-6f1a965ccb67 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:49.558702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-006ce6ca-239e-4286-956b-6f1a965ccb67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:49.572461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce6ca-239e-4286-956b-6f1a965ccb67 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:49.578080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:49.578474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:49.578750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:49.579559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] PATH_INFO: `/v3/projects/2d12a9aff428415f8618a620e135a41e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:49.582266 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d12a9aff428415f8618a620e135a41e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:49.582370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:49.693721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d12a9aff428415f8618a620e135a41e', 'request_id': 'req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:49.700409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d12a9aff428415f8618a620e135a41e', 'request_id': 'req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:49.853161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d12a9aff428415f8618a620e135a41e', 'request_id': 'req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:49.871397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] The token cache is being invalidate because project 2d12a9aff428415f8618a620e135a41e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:43:49.871809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:49.873626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f43e53b8-c89c-4fa4-bbb3-1582abb64a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:43:49.876094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 997/2028] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:43:49 2022] DELETE /identity/v3/projects/2d12a9aff428415f8618a620e135a41e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:49.881012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5b0b1-faaa-4806-aab0-5ec3fad24b6b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:49.881530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5b0b1-faaa-4806-aab0-5ec3fad24b6b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:49.881530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5b0b1-faaa-4806-aab0-5ec3fad24b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:49.922632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-60f5b0b1-faaa-4806-aab0-5ec3fad24b6b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:49.923894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-60f5b0b1-faaa-4806-aab0-5ec3fad24b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:49.988048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1032/2029] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:49.996004 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c221-73c1-4301-b0e5-be67637b24fa None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:49.996266 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c221-73c1-4301-b0e5-be67637b24fa None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:49.996537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c221-73c1-4301-b0e5-be67637b24fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:50.016253 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b603c221-73c1-4301-b0e5-be67637b24fa None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:50.017933 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b603c221-73c1-4301-b0e5-be67637b24fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:50.026210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 998/2030] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:50.030618 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5b0b1-faaa-4806-aab0-5ec3fad24b6b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:50.041067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5b0b1-faaa-4806-aab0-5ec3fad24b6b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:50.045720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60e866-2272-4908-8616-9ad26d0a5085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:50.046352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60e866-2272-4908-8616-9ad26d0a5085 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:50.046640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60e866-2272-4908-8616-9ad26d0a5085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:50.046914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60e866-2272-4908-8616-9ad26d0a5085 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:50.047561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60e866-2272-4908-8616-9ad26d0a5085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:43:50.047866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60e866-2272-4908-8616-9ad26d0a5085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:50.049522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60e866-2272-4908-8616-9ad26d0a5085 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:50.062332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1033/2031] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:43:50.065926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603c221-73c1-4301-b0e5-be67637b24fa None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:50.075320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603c221-73c1-4301-b0e5-be67637b24fa None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:50.080545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15433b-8ec1-4216-9a36-f2693bed9187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:50.081151 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15433b-8ec1-4216-9a36-f2693bed9187 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:50.081430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15433b-8ec1-4216-9a36-f2693bed9187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:50.081696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15433b-8ec1-4216-9a36-f2693bed9187 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:50.082362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15433b-8ec1-4216-9a36-f2693bed9187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:50.084334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15433b-8ec1-4216-9a36-f2693bed9187 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:50.196602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:50.203623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 999/2032] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:43:50 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:43:50.209826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60e866-2272-4908-8616-9ad26d0a5085 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:50.220805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60e866-2272-4908-8616-9ad26d0a5085 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:50.284964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f607d-301f-40b7-be93-f9e3346894bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:50.285686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f607d-301f-40b7-be93-f9e3346894bc admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:50.285966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f607d-301f-40b7-be93-f9e3346894bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:50.286356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f607d-301f-40b7-be93-f9e3346894bc admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:50.286980 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f607d-301f-40b7-be93-f9e3346894bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:50.289176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f607d-301f-40b7-be93-f9e3346894bc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:50.412858 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1034/2033] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:43:50 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:43:50.417164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15433b-8ec1-4216-9a36-f2693bed9187 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:50.429146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15433b-8ec1-4216-9a36-f2693bed9187 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:50.434287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbe15e-746e-43e5-97a1-344f3f977b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:50.434962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbe15e-746e-43e5-97a1-344f3f977b5d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:50.435300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbe15e-746e-43e5-97a1-344f3f977b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:50.435586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbe15e-746e-43e5-97a1-344f3f977b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:50.436369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbe15e-746e-43e5-97a1-344f3f977b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:50.438253 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbe15e-746e-43e5-97a1-344f3f977b5d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:50.449272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1000/2034] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:50.453353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f607d-301f-40b7-be93-f9e3346894bc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:50.464204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f607d-301f-40b7-be93-f9e3346894bc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:50.469174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00a9ab-0f92-4170-b8b7-4e628f0d0875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:50.469782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a9ab-0f92-4170-b8b7-4e628f0d0875 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:50.470079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a9ab-0f92-4170-b8b7-4e628f0d0875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:50.470389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a9ab-0f92-4170-b8b7-4e628f0d0875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:50.471067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00a9ab-0f92-4170-b8b7-4e628f0d0875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:50.472689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00a9ab-0f92-4170-b8b7-4e628f0d0875 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:50.485564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1035/2035] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:43:50.489795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbe15e-746e-43e5-97a1-344f3f977b5d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:50.502673 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbe15e-746e-43e5-97a1-344f3f977b5d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:50.507571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ceae8d-d45a-417b-8ff0-34a7844a8276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:50.508262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ceae8d-d45a-417b-8ff0-34a7844a8276 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:50.508511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ceae8d-d45a-417b-8ff0-34a7844a8276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:50.508781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ceae8d-d45a-417b-8ff0-34a7844a8276 admin admin] PATH_INFO: `/v3/projects/318c2ab91f564d8ba2e29b724c008dbc/users/cadc251db5cf4e24a3fc8ffab6e9b9ae/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:50.551523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ceae8d-d45a-417b-8ff0-34a7844a8276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318c2ab91f564d8ba2e29b724c008dbc, user_id=cadc251db5cf4e24a3fc8ffab6e9b9ae, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:50.554061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ceae8d-d45a-417b-8ff0-34a7844a8276 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:50.642980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1001/2036] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:43:50 2022] PUT /identity/v3/projects/318c2ab91f564d8ba2e29b724c008dbc/users/cadc251db5cf4e24a3fc8ffab6e9b9ae/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:43:50.657300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bc29f-e08f-4c93-8fbe-59516a2c2cb8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:50.657619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bc29f-e08f-4c93-8fbe-59516a2c2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:50.657892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bc29f-e08f-4c93-8fbe-59516a2c2cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:50.689992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5e6bc29f-e08f-4c93-8fbe-59516a2c2cb8 None None] MFA Rules not processed for user `cadc251db5cf4e24a3fc8ffab6e9b9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:50.691291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5e6bc29f-e08f-4c93-8fbe-59516a2c2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:50.736020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:43:50.762726 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1036/2037] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:50.790439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ceae8d-d45a-417b-8ff0-34a7844a8276 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:50.792313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e6ceae8d-d45a-417b-8ff0-34a7844a8276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:50.878534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ceae8d-d45a-417b-8ff0-34a7844a8276 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:50.882651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bf2cc-3421-422a-b300-ceccb8a3eb4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:50.883265 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bf2cc-3421-422a-b300-ceccb8a3eb4b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:50.883550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bf2cc-3421-422a-b300-ceccb8a3eb4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:50.883824 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bf2cc-3421-422a-b300-ceccb8a3eb4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:50.902459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6667dbe-57f3-4398-8f97-f3ec939e94a1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:50.903622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6667dbe-57f3-4398-8f97-f3ec939e94a1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:50.904023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6667dbe-57f3-4398-8f97-f3ec939e94a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:50.906407 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bf2cc-3421-422a-b300-ceccb8a3eb4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:50.911063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bf2cc-3421-422a-b300-ceccb8a3eb4b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:50.922893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1002/2038] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:50.939682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e6667dbe-57f3-4398-8f97-f3ec939e94a1 None None] MFA Rules not processed for user `877575c9e0264241b9eab868a89cbbe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:50.941157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e6667dbe-57f3-4398-8f97-f3ec939e94a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:51.057377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1037/2039] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:43:51.145761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bf2cc-3421-422a-b300-ceccb8a3eb4b service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:51.147189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-259bf2cc-3421-422a-b300-ceccb8a3eb4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:51.158926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bf2cc-3421-422a-b300-ceccb8a3eb4b service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:51.165436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d209a-7540-4108-bebb-80c4d19504ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:51.165436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d209a-7540-4108-bebb-80c4d19504ff service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:51.165436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d209a-7540-4108-bebb-80c4d19504ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:51.165436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d209a-7540-4108-bebb-80c4d19504ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:51.172124 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d209a-7540-4108-bebb-80c4d19504ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:51.174223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d209a-7540-4108-bebb-80c4d19504ff service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:51.183541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1003/2040] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:43:54.369588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb33e2-67f3-4d0f-a19f-ca4a06607fd6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:54.370116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb33e2-67f3-4d0f-a19f-ca4a06607fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:54.370116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb33e2-67f3-4d0f-a19f-ca4a06607fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:54.389624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5cbb33e2-67f3-4d0f-a19f-ca4a06607fd6 None None] MFA Rules not processed for user `cadc251db5cf4e24a3fc8ffab6e9b9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:54.390944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5cbb33e2-67f3-4d0f-a19f-ca4a06607fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:54.397970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1038/2041] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:54.512828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de962d-45cb-433f-933c-3e95a3346fb2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:54.513386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de962d-45cb-433f-933c-3e95a3346fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:54.513386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de962d-45cb-433f-933c-3e95a3346fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:54.532664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-15de962d-45cb-433f-933c-3e95a3346fb2 None None] MFA Rules not processed for user `877575c9e0264241b9eab868a89cbbe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:43:54.533919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-15de962d-45cb-433f-933c-3e95a3346fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:54.540831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1004/2042] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:43:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:43:55.362732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb33e2-67f3-4d0f-a19f-ca4a06607fd6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:43:55.375730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb33e2-67f3-4d0f-a19f-ca4a06607fd6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:43:55.381974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d77d0-d8e9-40b7-9a0a-ae26405bd63c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:43:55.382728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d77d0-d8e9-40b7-9a0a-ae26405bd63c service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:43:55.383107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d77d0-d8e9-40b7-9a0a-ae26405bd63c service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:43:55.383438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d77d0-d8e9-40b7-9a0a-ae26405bd63c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:43:55.385462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-418d77d0-d8e9-40b7-9a0a-ae26405bd63c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:43:55.453449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d77d0-d8e9-40b7-9a0a-ae26405bd63c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:43:55.454939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d77d0-d8e9-40b7-9a0a-ae26405bd63c service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:43:55.465404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1039/2043] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:43:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:00.173967 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de962d-45cb-433f-933c-3e95a3346fb2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:00.175597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-15de962d-45cb-433f-933c-3e95a3346fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:00.242840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de962d-45cb-433f-933c-3e95a3346fb2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:00.247809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:00.248433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:00.248708 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:00.248975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] PATH_INFO: `/v3/users/e17e82fab7c245d3ad003dead541839d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:00.250089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17e82fab7c245d3ad003dead541839d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:00.252075 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:00.471694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17e82fab7c245d3ad003dead541839d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:00.472350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17e82fab7c245d3ad003dead541839d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:00.474933 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:00.485246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17e82fab7c245d3ad003dead541839d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:00.611687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17e82fab7c245d3ad003dead541839d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:00.612832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17e82fab7c245d3ad003dead541839d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:00.625910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1005/2044] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:00 2022] DELETE /identity/v3/users/e17e82fab7c245d3ad003dead541839d => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:00.630080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7727a2-8457-4ac3-8904-311aa1d49e36 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:00.630584 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7727a2-8457-4ac3-8904-311aa1d49e36 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:00.630676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7727a2-8457-4ac3-8904-311aa1d49e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:00.653156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6d7727a2-8457-4ac3-8904-311aa1d49e36 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:00.654572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6d7727a2-8457-4ac3-8904-311aa1d49e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:00.718091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:00.727465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1040/2045] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:00.750878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:00.752434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:00.819529 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c2047-ad4b-44c2-963d-0349f5e6f33d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:00.823451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d364f13-7615-4f25-8f47-537ecf239cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:00.824071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d364f13-7615-4f25-8f47-537ecf239cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:00.825072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d364f13-7615-4f25-8f47-537ecf239cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:00.825072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d364f13-7615-4f25-8f47-537ecf239cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:00.837556 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d364f13-7615-4f25-8f47-537ecf239cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:00.839323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d364f13-7615-4f25-8f47-537ecf239cc8 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:00.853494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1006/2046] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:44:01.356827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7727a2-8457-4ac3-8904-311aa1d49e36 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:01.358697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6d7727a2-8457-4ac3-8904-311aa1d49e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:01.371125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7727a2-8457-4ac3-8904-311aa1d49e36 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:01.376022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:01.377416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:01.377416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:01.377416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] PATH_INFO: `/v3/projects/166224e84d534e25ab1ca7df3380034a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:01.378952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=166224e84d534e25ab1ca7df3380034a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:01.380905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:01.511103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '166224e84d534e25ab1ca7df3380034a', 'request_id': 'req-df3e2acd-11d4-4cf2-92a2-50d8222adf11'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:01.523567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '166224e84d534e25ab1ca7df3380034a', 'request_id': 'req-df3e2acd-11d4-4cf2-92a2-50d8222adf11'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:01.583415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '166224e84d534e25ab1ca7df3380034a', 'request_id': 'req-df3e2acd-11d4-4cf2-92a2-50d8222adf11'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:01.593985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] The token cache is being invalidate because project 166224e84d534e25ab1ca7df3380034a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:01.594289 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:01.594989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:01.596614 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1041/2047] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:01 2022] DELETE /identity/v3/projects/166224e84d534e25ab1ca7df3380034a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:01.600676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d364f13-7615-4f25-8f47-537ecf239cc8 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:01.610332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d364f13-7615-4f25-8f47-537ecf239cc8 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:01.664922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:01.665547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:01.665831 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:01.666104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] PATH_INFO: `/v3/users/5851db4f7c6c4990baae7c9cb27b27b5` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:01.667231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5851db4f7c6c4990baae7c9cb27b27b5)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:01.669141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:01.873489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5851db4f7c6c4990baae7c9cb27b27b5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:01.874151 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5851db4f7c6c4990baae7c9cb27b27b5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:01.887110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5851db4f7c6c4990baae7c9cb27b27b5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:01.948698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5851db4f7c6c4990baae7c9cb27b27b5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:01.949323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5851db4f7c6c4990baae7c9cb27b27b5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:01.950796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:01.962823 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1007/2048] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:01 2022] DELETE /identity/v3/users/5851db4f7c6c4990baae7c9cb27b27b5 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:02.401541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:02.403055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:02.418110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e2acd-11d4-4cf2-92a2-50d8222adf11 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:02.474869 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:02.475494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:02.475770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:02.476039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] PATH_INFO: `/v3/projects/d84f33237945468cb3c659eb9e5025e2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:02.477149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d84f33237945468cb3c659eb9e5025e2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:02.479101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:02.589197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd84f33237945468cb3c659eb9e5025e2', 'request_id': 'req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:02.595778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd84f33237945468cb3c659eb9e5025e2', 'request_id': 'req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:02.651942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd84f33237945468cb3c659eb9e5025e2', 'request_id': 'req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:02.664410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] The token cache is being invalidate because project d84f33237945468cb3c659eb9e5025e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:02.664704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:02.665412 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:02.667147 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1042/2049] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:02 2022] DELETE /identity/v3/projects/d84f33237945468cb3c659eb9e5025e2 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:02.671010 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:02.672514 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:02.685426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3808b-663d-4a5d-b92a-1478c47edda6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:02.746447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:02.749111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:02.749111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:02.749111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] PATH_INFO: `/v3/users/77dd0152b25b4e77864185b385671d72` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:02.749111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=77dd0152b25b4e77864185b385671d72)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:02.750685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:02.966926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77dd0152b25b4e77864185b385671d72'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:02.967508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77dd0152b25b4e77864185b385671d72'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:02.980483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77dd0152b25b4e77864185b385671d72'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:03.052299 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77dd0152b25b4e77864185b385671d72'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:03.053050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-1c5f5c26-ced4-4a83-9c50-d8685174d1ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77dd0152b25b4e77864185b385671d72'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:03.066796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1008/2050] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:02 2022] DELETE /identity/v3/users/77dd0152b25b4e77864185b385671d72 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:03.589743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:03.591476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:03.613625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833b3ea-fd0f-483b-9cfe-8abaa74f6e3f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:03.643842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:03.712139 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:03.712790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:03.713056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:03.713326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] PATH_INFO: `/v3/projects/2c83dd71632b4ea09876f14a39ff6132` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:03.714927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c83dd71632b4ea09876f14a39ff6132)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:03.716825 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:03.882850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c83dd71632b4ea09876f14a39ff6132', 'request_id': 'req-e147b673-6527-4152-8996-ba3125403879'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:03.905925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c83dd71632b4ea09876f14a39ff6132', 'request_id': 'req-e147b673-6527-4152-8996-ba3125403879'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:03.993016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c83dd71632b4ea09876f14a39ff6132', 'request_id': 'req-e147b673-6527-4152-8996-ba3125403879'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:04.007867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] The token cache is being invalidate because project 2c83dd71632b4ea09876f14a39ff6132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:04.008265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:04.009090 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e147b673-6527-4152-8996-ba3125403879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:04.012285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1043/2051] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:03 2022] DELETE /identity/v3/projects/2c83dd71632b4ea09876f14a39ff6132 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:04.022418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e004242-bc6b-4346-817f-bb80928c4966 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:04.022941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e004242-bc6b-4346-817f-bb80928c4966 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:04.023019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e004242-bc6b-4346-817f-bb80928c4966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:04.047716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9e004242-bc6b-4346-817f-bb80928c4966 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:04.048950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9e004242-bc6b-4346-817f-bb80928c4966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:04.128166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1009/2052] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:04.130648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de27be0-743f-48f9-b3b4-012f61523ebb None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:04.130978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de27be0-743f-48f9-b3b4-012f61523ebb None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:04.131497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de27be0-743f-48f9-b3b4-012f61523ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:04.156373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6de27be0-743f-48f9-b3b4-012f61523ebb None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:04.157627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6de27be0-743f-48f9-b3b4-012f61523ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:04.176305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1044/2053] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:04.177777 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e004242-bc6b-4346-817f-bb80928c4966 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:04.188920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e004242-bc6b-4346-817f-bb80928c4966 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:04.194324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2341-7ca3-41c4-a4e1-de48fa1b371b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:04.195011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2341-7ca3-41c4-a4e1-de48fa1b371b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:04.195287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2341-7ca3-41c4-a4e1-de48fa1b371b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:04.195585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2341-7ca3-41c4-a4e1-de48fa1b371b admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:04.196380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f2341-7ca3-41c4-a4e1-de48fa1b371b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:04.196684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f2341-7ca3-41c4-a4e1-de48fa1b371b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:04.198289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f2341-7ca3-41c4-a4e1-de48fa1b371b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:04.201168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:04.213338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1010/2054] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:44:04.217606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de27be0-743f-48f9-b3b4-012f61523ebb None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:04.231296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de27be0-743f-48f9-b3b4-012f61523ebb None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:04.236719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdead0-2093-4079-86d6-6129dd9e014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:04.237353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdead0-2093-4079-86d6-6129dd9e014e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:04.237627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdead0-2093-4079-86d6-6129dd9e014e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:04.237901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdead0-2093-4079-86d6-6129dd9e014e admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:04.238577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdead0-2093-4079-86d6-6129dd9e014e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:04.240530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdead0-2093-4079-86d6-6129dd9e014e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:04.375967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1045/2055] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:04 2022] POST /identity/v3/projects => generated 403 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:04.387829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2341-7ca3-41c4-a4e1-de48fa1b371b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:04.398287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2341-7ca3-41c4-a4e1-de48fa1b371b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:04.454481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8affbf-ff54-4686-af45-2b830a397bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:04.455143 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8affbf-ff54-4686-af45-2b830a397bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:04.455420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8affbf-ff54-4686-af45-2b830a397bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:04.456322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8affbf-ff54-4686-af45-2b830a397bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:04.457180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8affbf-ff54-4686-af45-2b830a397bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:04.459443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8affbf-ff54-4686-af45-2b830a397bf4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:04.546540 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1011/2056] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:04 2022] POST /identity/v3/users => generated 302 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:04.547789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdead0-2093-4079-86d6-6129dd9e014e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:04.558671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdead0-2093-4079-86d6-6129dd9e014e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:04.563532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84975e14-c2b2-49f2-9ce6-f11f04d0abea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:04.564125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84975e14-c2b2-49f2-9ce6-f11f04d0abea admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:04.564398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84975e14-c2b2-49f2-9ce6-f11f04d0abea admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:04.564664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84975e14-c2b2-49f2-9ce6-f11f04d0abea admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:04.565827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84975e14-c2b2-49f2-9ce6-f11f04d0abea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:04.567676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84975e14-c2b2-49f2-9ce6-f11f04d0abea admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:04.577865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1046/2057] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:04.582925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8affbf-ff54-4686-af45-2b830a397bf4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:04.593051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8affbf-ff54-4686-af45-2b830a397bf4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:04.597841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731dcf65-2709-4a3e-9185-b7f57a9399a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:04.598494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dcf65-2709-4a3e-9185-b7f57a9399a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:04.598771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dcf65-2709-4a3e-9185-b7f57a9399a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:04.599044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dcf65-2709-4a3e-9185-b7f57a9399a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:04.599689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731dcf65-2709-4a3e-9185-b7f57a9399a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:04.601309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731dcf65-2709-4a3e-9185-b7f57a9399a4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:04.612451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1012/2058] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:04.616642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84975e14-c2b2-49f2-9ce6-f11f04d0abea admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:04.626920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84975e14-c2b2-49f2-9ce6-f11f04d0abea admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:04.632105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412464f6-3c3d-4774-bde5-623a78d72db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:04.632757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412464f6-3c3d-4774-bde5-623a78d72db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:04.633056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412464f6-3c3d-4774-bde5-623a78d72db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:04.633314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412464f6-3c3d-4774-bde5-623a78d72db8 admin admin] PATH_INFO: `/v3/projects/7ca58828350643f8a8194369351c590d/users/c18a4de4a5d14a99b4165d55c4a9caf7/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:04.648630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412464f6-3c3d-4774-bde5-623a78d72db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca58828350643f8a8194369351c590d, user_id=c18a4de4a5d14a99b4165d55c4a9caf7, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:04.651361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412464f6-3c3d-4774-bde5-623a78d72db8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:04.751797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1047/2059] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:04 2022] PUT /identity/v3/projects/7ca58828350643f8a8194369351c590d/users/c18a4de4a5d14a99b4165d55c4a9caf7/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:04.763617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b06363-159b-48b6-9447-9ed10df466d4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:04.764083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b06363-159b-48b6-9447-9ed10df466d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:04.764429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b06363-159b-48b6-9447-9ed10df466d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:04.798065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-74b06363-159b-48b6-9447-9ed10df466d4 None None] MFA Rules not processed for user `c18a4de4a5d14a99b4165d55c4a9caf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:04.800112 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-74b06363-159b-48b6-9447-9ed10df466d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:04.870081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1013/2060] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:04.875520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412464f6-3c3d-4774-bde5-623a78d72db8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:04.918952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:04.938978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412464f6-3c3d-4774-bde5-623a78d72db8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:04.944697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073b2f3-12e7-4a72-94f5-f9bb67c47eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:04.945306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073b2f3-12e7-4a72-94f5-f9bb67c47eca admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:04.945587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073b2f3-12e7-4a72-94f5-f9bb67c47eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:04.945861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073b2f3-12e7-4a72-94f5-f9bb67c47eca admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:04.946556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073b2f3-12e7-4a72-94f5-f9bb67c47eca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:04.948513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073b2f3-12e7-4a72-94f5-f9bb67c47eca admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:05.048575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1048/2061] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:04 2022] POST /identity/v3/projects => generated 409 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:05.052717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b06363-159b-48b6-9447-9ed10df466d4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:05.062630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b06363-159b-48b6-9447-9ed10df466d4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:05.129235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137ef26-8421-48c1-a761-49ca3c985db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:05.129869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137ef26-8421-48c1-a761-49ca3c985db0 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:05.130161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137ef26-8421-48c1-a761-49ca3c985db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:05.130457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137ef26-8421-48c1-a761-49ca3c985db0 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:05.131093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137ef26-8421-48c1-a761-49ca3c985db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:05.133027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137ef26-8421-48c1-a761-49ca3c985db0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:05.143546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:05.250314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1014/2062] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:05 2022] POST /identity/v3/users => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:05.254190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073b2f3-12e7-4a72-94f5-f9bb67c47eca admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:05.266129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073b2f3-12e7-4a72-94f5-f9bb67c47eca admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:05.271035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78fe18-d0d4-46f8-9c4e-006fb58d6bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:05.271649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78fe18-d0d4-46f8-9c4e-006fb58d6bdf admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:05.271973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78fe18-d0d4-46f8-9c4e-006fb58d6bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:05.272244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78fe18-d0d4-46f8-9c4e-006fb58d6bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:05.272902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78fe18-d0d4-46f8-9c4e-006fb58d6bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:05.274521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78fe18-d0d4-46f8-9c4e-006fb58d6bdf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:05.289239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1049/2063] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:05.290343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137ef26-8421-48c1-a761-49ca3c985db0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:05.301105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137ef26-8421-48c1-a761-49ca3c985db0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:05.307082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e61df-a07a-4aeb-a197-91a82d8c7d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:05.307754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e61df-a07a-4aeb-a197-91a82d8c7d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:05.308038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e61df-a07a-4aeb-a197-91a82d8c7d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:05.308312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e61df-a07a-4aeb-a197-91a82d8c7d44 admin admin] PATH_INFO: `/v3/projects/15e70527aeb14f6bbe146768f91e3ff2/users/9fe9f43219144ec2af347356ed2f8b58/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:05.328077 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e61df-a07a-4aeb-a197-91a82d8c7d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e70527aeb14f6bbe146768f91e3ff2, user_id=9fe9f43219144ec2af347356ed2f8b58, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:05.334513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e61df-a07a-4aeb-a197-91a82d8c7d44 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:05.440488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1015/2064] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:05 2022] PUT /identity/v3/projects/15e70527aeb14f6bbe146768f91e3ff2/users/9fe9f43219144ec2af347356ed2f8b58/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:05.440676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78fe18-d0d4-46f8-9c4e-006fb58d6bdf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:05.467523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78fe18-d0d4-46f8-9c4e-006fb58d6bdf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:05.544733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c929e-ad06-490e-8cb4-15c5ae62a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:05.546086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c929e-ad06-490e-8cb4-15c5ae62a47a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:05.546472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c929e-ad06-490e-8cb4-15c5ae62a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:05.546822 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c929e-ad06-490e-8cb4-15c5ae62a47a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:05.547894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c929e-ad06-490e-8cb4-15c5ae62a47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:05.549762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c929e-ad06-490e-8cb4-15c5ae62a47a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:05.564207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1050/2065] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:05.585172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e61df-a07a-4aeb-a197-91a82d8c7d44 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:05.594168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e61df-a07a-4aeb-a197-91a82d8c7d44 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:05.600922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13df25e-fa09-4c78-9da0-116f19fe043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:05.601687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13df25e-fa09-4c78-9da0-116f19fe043f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:05.602025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13df25e-fa09-4c78-9da0-116f19fe043f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:05.602679 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13df25e-fa09-4c78-9da0-116f19fe043f admin admin] PATH_INFO: `/v3/projects/15e70527aeb14f6bbe146768f91e3ff2/users/9fe9f43219144ec2af347356ed2f8b58/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:05.604788 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13df25e-fa09-4c78-9da0-116f19fe043f admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e70527aeb14f6bbe146768f91e3ff2, user_id=9fe9f43219144ec2af347356ed2f8b58, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:05.607755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13df25e-fa09-4c78-9da0-116f19fe043f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:05.626552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1016/2066] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:05 2022] PUT /identity/v3/projects/15e70527aeb14f6bbe146768f91e3ff2/users/9fe9f43219144ec2af347356ed2f8b58/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:05.634173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e2815-9360-42ca-beb8-f128ce869963 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:05.634665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e2815-9360-42ca-beb8-f128ce869963 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:05.634756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e2815-9360-42ca-beb8-f128ce869963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:05.669130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-244e2815-9360-42ca-beb8-f128ce869963 None None] MFA Rules not processed for user `9fe9f43219144ec2af347356ed2f8b58`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:05.670665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-244e2815-9360-42ca-beb8-f128ce869963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:05.764265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1051/2067] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:05 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:05.778201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13df25e-fa09-4c78-9da0-116f19fe043f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:05.783478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e13df25e-fa09-4c78-9da0-116f19fe043f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:05.931946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13df25e-fa09-4c78-9da0-116f19fe043f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:05.944889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf1910-a748-4196-a03b-d758231d70bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:05.945684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf1910-a748-4196-a03b-d758231d70bc service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:05.945972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf1910-a748-4196-a03b-d758231d70bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:05.946286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf1910-a748-4196-a03b-d758231d70bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:05.973660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf1910-a748-4196-a03b-d758231d70bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:05.975483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf1910-a748-4196-a03b-d758231d70bc service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:05.990028 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1017/2068] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:05 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:44:06.425794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e2815-9360-42ca-beb8-f128ce869963 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:06.435490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e2815-9360-42ca-beb8-f128ce869963 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:06.439329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d25e95-d660-4671-8369-e97424d1ee26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:06.440469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d25e95-d660-4671-8369-e97424d1ee26 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:06.440751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d25e95-d660-4671-8369-e97424d1ee26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:06.441024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d25e95-d660-4671-8369-e97424d1ee26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:06.473620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:06.497899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d25e95-d660-4671-8369-e97424d1ee26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:06.499370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d25e95-d660-4671-8369-e97424d1ee26 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:06.508813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1052/2069] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:06 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:07.874738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf1910-a748-4196-a03b-d758231d70bc service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:07.876326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1fdf1910-a748-4196-a03b-d758231d70bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:07.942975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf1910-a748-4196-a03b-d758231d70bc service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:07.947709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:07.948372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:07.948646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:07.948912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] PATH_INFO: `/v3/users/36d4e8e9ecf94e89a37161032689c4fe` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:07.950037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d4e8e9ecf94e89a37161032689c4fe)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:07.951986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:08.156873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d4e8e9ecf94e89a37161032689c4fe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:08.157559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d4e8e9ecf94e89a37161032689c4fe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:08.169287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:08.172684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d4e8e9ecf94e89a37161032689c4fe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:08.251834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d4e8e9ecf94e89a37161032689c4fe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:08.252526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-528976a6-4a9b-4c41-99a2-709dc71cc0cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d4e8e9ecf94e89a37161032689c4fe'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:08.266736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d25e95-d660-4671-8369-e97424d1ee26 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:08.268024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1018/2070] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:07 2022] DELETE /identity/v3/users/36d4e8e9ecf94e89a37161032689c4fe => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:08.268364 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-44d25e95-d660-4671-8369-e97424d1ee26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:08.272706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fdae5-125a-4ad9-aa33-fec5ae719c8b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:08.273410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fdae5-125a-4ad9-aa33-fec5ae719c8b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:08.273695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fdae5-125a-4ad9-aa33-fec5ae719c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:08.284942 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d25e95-d660-4671-8369-e97424d1ee26 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:08.296879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-281fdae5-125a-4ad9-aa33-fec5ae719c8b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:08.299085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-281fdae5-125a-4ad9-aa33-fec5ae719c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:08.341505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:08.342199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:08.342479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:08.342755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] PATH_INFO: `/v3/users/9d1d5de04c894a2ab780bcdee15f5c5b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:08.343884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d1d5de04c894a2ab780bcdee15f5c5b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:08.345797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:08.359460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1019/2071] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:08.369559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fdae5-125a-4ad9-aa33-fec5ae719c8b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:08.371015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-281fdae5-125a-4ad9-aa33-fec5ae719c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:08.428638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fdae5-125a-4ad9-aa33-fec5ae719c8b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:08.432332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fd10b-23cd-4058-8ff0-7ac47cb84477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:08.432938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fd10b-23cd-4058-8ff0-7ac47cb84477 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:08.433215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fd10b-23cd-4058-8ff0-7ac47cb84477 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:08.433610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fd10b-23cd-4058-8ff0-7ac47cb84477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:08.443710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fd10b-23cd-4058-8ff0-7ac47cb84477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:08.445134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fd10b-23cd-4058-8ff0-7ac47cb84477 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:08.457059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1020/2072] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:44:08.519242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d1d5de04c894a2ab780bcdee15f5c5b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:08.537204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d1d5de04c894a2ab780bcdee15f5c5b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:08.538607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d1d5de04c894a2ab780bcdee15f5c5b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:08.605777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d1d5de04c894a2ab780bcdee15f5c5b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:08.606570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d1d5de04c894a2ab780bcdee15f5c5b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:08.621360 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1053/2073] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:08 2022] DELETE /identity/v3/users/9d1d5de04c894a2ab780bcdee15f5c5b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:09.025705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fd10b-23cd-4058-8ff0-7ac47cb84477 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:09.029300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-001fd10b-23cd-4058-8ff0-7ac47cb84477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:09.030181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:09.033089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:09.048265 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fd10b-23cd-4058-8ff0-7ac47cb84477 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:09.109224 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:09.122935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e76aff-31d3-4b11-ae0d-f1fc5fea6bf1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:09.127135 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:09.128140 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:09.128277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:09.128574 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:09.128774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:09.128877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] PATH_INFO: `/v3/projects/ec57e46b298a471b9267e04a93333a9c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:09.129110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:09.129387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] PATH_INFO: `/v3/projects/b427232479e14a6fa576f4461032f12b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:09.129970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec57e46b298a471b9267e04a93333a9c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:09.130632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b427232479e14a6fa576f4461032f12b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:09.131907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:09.132583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:09.242731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec57e46b298a471b9267e04a93333a9c', 'request_id': 'req-7f5b4618-d8cf-4c09-8854-90b1f66559da'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.248873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec57e46b298a471b9267e04a93333a9c', 'request_id': 'req-7f5b4618-d8cf-4c09-8854-90b1f66559da'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.271523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b427232479e14a6fa576f4461032f12b', 'request_id': 'req-dd8d566c-b749-44ca-966e-a50efad5239a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.278127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b427232479e14a6fa576f4461032f12b', 'request_id': 'req-dd8d566c-b749-44ca-966e-a50efad5239a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.357174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec57e46b298a471b9267e04a93333a9c', 'request_id': 'req-7f5b4618-d8cf-4c09-8854-90b1f66559da'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.369061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] The token cache is being invalidate because project ec57e46b298a471b9267e04a93333a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:09.369357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.370030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.371588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1021/2074] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:09 2022] DELETE /identity/v3/projects/ec57e46b298a471b9267e04a93333a9c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:09.375516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:09.376891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:09.442694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b4618-d8cf-4c09-8854-90b1f66559da admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:09.459550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b427232479e14a6fa576f4461032f12b', 'request_id': 'req-dd8d566c-b749-44ca-966e-a50efad5239a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.470832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] The token cache is being invalidate because project b427232479e14a6fa576f4461032f12b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:09.471123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.471756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dd8d566c-b749-44ca-966e-a50efad5239a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.473276 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1054/2075] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:09 2022] DELETE /identity/v3/projects/b427232479e14a6fa576f4461032f12b => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:09.478079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d303f-69ed-4382-a261-a8bd8765b080 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:09.478384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d303f-69ed-4382-a261-a8bd8765b080 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:09.478656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d303f-69ed-4382-a261-a8bd8765b080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:09.497944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d00d303f-69ed-4382-a261-a8bd8765b080 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:09.498318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:09.498957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:09.499169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d00d303f-69ed-4382-a261-a8bd8765b080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:09.499291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:09.499561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] PATH_INFO: `/v3/users/4f4140758779488ba28b54a527031c76` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:09.500800 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f4140758779488ba28b54a527031c76)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:09.503348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:09.507688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1055/2076] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:09.515164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41166e0-6a71-485a-91d6-f895b2fae38c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:09.515447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41166e0-6a71-485a-91d6-f895b2fae38c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:09.515739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41166e0-6a71-485a-91d6-f895b2fae38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:09.534686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b41166e0-6a71-485a-91d6-f895b2fae38c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:09.535801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b41166e0-6a71-485a-91d6-f895b2fae38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:09.543633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1056/2077] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:09.547559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41166e0-6a71-485a-91d6-f895b2fae38c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:09.556713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41166e0-6a71-485a-91d6-f895b2fae38c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:09.561277 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f905321-f80b-4c5b-9668-facf40f88660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:09.568221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f905321-f80b-4c5b-9668-facf40f88660 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:09.568221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f905321-f80b-4c5b-9668-facf40f88660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:09.568221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f905321-f80b-4c5b-9668-facf40f88660 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:09.568221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f905321-f80b-4c5b-9668-facf40f88660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:09.568221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f905321-f80b-4c5b-9668-facf40f88660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:09.568221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f905321-f80b-4c5b-9668-facf40f88660 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:09.577399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1057/2078] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:44:09.581663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f905321-f80b-4c5b-9668-facf40f88660 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:09.591093 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f905321-f80b-4c5b-9668-facf40f88660 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:09.595648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ea063-622c-4469-ac46-663b97c98e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:09.597568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ea063-622c-4469-ac46-663b97c98e50 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:09.597847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ea063-622c-4469-ac46-663b97c98e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:09.598119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ea063-622c-4469-ac46-663b97c98e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:09.598824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ea063-622c-4469-ac46-663b97c98e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:09.600714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ea063-622c-4469-ac46-663b97c98e50 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:09.690608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1058/2079] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:09 2022] POST /identity/v3/projects => generated 393 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:44:09.691911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ea063-622c-4469-ac46-663b97c98e50 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:09.725442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f4140758779488ba28b54a527031c76'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.726113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f4140758779488ba28b54a527031c76'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.741589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f4140758779488ba28b54a527031c76'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.757443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ea063-622c-4469-ac46-663b97c98e50 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:09.762216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f6928-1d2a-4d31-8658-15773d7a84d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:09.762822 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f6928-1d2a-4d31-8658-15773d7a84d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:09.763103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f6928-1d2a-4d31-8658-15773d7a84d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:09.763378 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f6928-1d2a-4d31-8658-15773d7a84d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:09.764014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f6928-1d2a-4d31-8658-15773d7a84d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:09.765938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f6928-1d2a-4d31-8658-15773d7a84d9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:09.840386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f4140758779488ba28b54a527031c76'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.841001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f4140758779488ba28b54a527031c76'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:09.854165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1022/2080] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:09 2022] DELETE /identity/v3/users/4f4140758779488ba28b54a527031c76 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:09.898123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:09.898795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1059/2081] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:09 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:44:09.902852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:09.904302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:09.982765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e840561-4c4e-49ef-90c0-e6e36e582a59 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:09.988368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f127e7-7d2a-48ea-82b1-b25f2c377366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:09.988969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f127e7-7d2a-48ea-82b1-b25f2c377366 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:09.989250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f127e7-7d2a-48ea-82b1-b25f2c377366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:09.989521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f127e7-7d2a-48ea-82b1-b25f2c377366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:09.990185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f127e7-7d2a-48ea-82b1-b25f2c377366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:09.991769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f127e7-7d2a-48ea-82b1-b25f2c377366 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:10.005802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1023/2082] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:10.006619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f6928-1d2a-4d31-8658-15773d7a84d9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:10.017718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f6928-1d2a-4d31-8658-15773d7a84d9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:10.025435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4191424-198e-41b6-95b1-0ee3d4e5c9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:10.025751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4191424-198e-41b6-95b1-0ee3d4e5c9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:10.026029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4191424-198e-41b6-95b1-0ee3d4e5c9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:10.026335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4191424-198e-41b6-95b1-0ee3d4e5c9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:10.027010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4191424-198e-41b6-95b1-0ee3d4e5c9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:10.028745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4191424-198e-41b6-95b1-0ee3d4e5c9c8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:10.042442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1060/2083] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:10.046944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f127e7-7d2a-48ea-82b1-b25f2c377366 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:10.068609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f127e7-7d2a-48ea-82b1-b25f2c377366 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:10.071535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6837c0-373a-45c5-ba15-8a2cb28bda64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:10.072204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6837c0-373a-45c5-ba15-8a2cb28bda64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:10.072482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6837c0-373a-45c5-ba15-8a2cb28bda64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:10.072798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6837c0-373a-45c5-ba15-8a2cb28bda64 admin admin] PATH_INFO: `/v3/projects/689c4759cb5943b79f2637a8e29e230e/users/c1b48a973c20470a9af23c77e3da5b72/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:10.088908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6837c0-373a-45c5-ba15-8a2cb28bda64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=689c4759cb5943b79f2637a8e29e230e, user_id=c1b48a973c20470a9af23c77e3da5b72, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:10.091911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6837c0-373a-45c5-ba15-8a2cb28bda64 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:10.166895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1024/2084] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:10 2022] PUT /identity/v3/projects/689c4759cb5943b79f2637a8e29e230e/users/c1b48a973c20470a9af23c77e3da5b72/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:10.172216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6f79b-765a-4c27-a1f9-3114f7ecff25 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:10.172503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6f79b-765a-4c27-a1f9-3114f7ecff25 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:10.172801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6f79b-765a-4c27-a1f9-3114f7ecff25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:10.191740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fff6f79b-765a-4c27-a1f9-3114f7ecff25 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:10.192940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fff6f79b-765a-4c27-a1f9-3114f7ecff25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:10.250784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1061/2085] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:10.260398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6837c0-373a-45c5-ba15-8a2cb28bda64 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:10.262163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-de6837c0-373a-45c5-ba15-8a2cb28bda64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:10.281828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6f79b-765a-4c27-a1f9-3114f7ecff25 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:10.283465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fff6f79b-765a-4c27-a1f9-3114f7ecff25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:10.296103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6f79b-765a-4c27-a1f9-3114f7ecff25 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:10.301142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:10.301783 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:10.302065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:10.302391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] PATH_INFO: `/v3/projects/6cbbc66b9a80441781fc83ccf5d82dfa` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:10.303692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cbbc66b9a80441781fc83ccf5d82dfa)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:10.305648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:10.326053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6837c0-373a-45c5-ba15-8a2cb28bda64 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:10.330052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecefdd-9d99-4a3e-b502-e5ada0576ed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:10.331304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecefdd-9d99-4a3e-b502-e5ada0576ed7 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:10.331586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecefdd-9d99-4a3e-b502-e5ada0576ed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:10.331924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecefdd-9d99-4a3e-b502-e5ada0576ed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:10.341086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecefdd-9d99-4a3e-b502-e5ada0576ed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:10.342627 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecefdd-9d99-4a3e-b502-e5ada0576ed7 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:10.352570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1025/2086] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:10.422697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cbbc66b9a80441781fc83ccf5d82dfa', 'request_id': 'req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:10.429015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cbbc66b9a80441781fc83ccf5d82dfa', 'request_id': 'req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:10.534012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cbbc66b9a80441781fc83ccf5d82dfa', 'request_id': 'req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:10.544685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] The token cache is being invalidate because project 6cbbc66b9a80441781fc83ccf5d82dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:10.544973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:10.545605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c9b10a6-4e3c-4a4d-95ce-bd456f7ec8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:10.547147 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1062/2087] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:10 2022] DELETE /identity/v3/projects/6cbbc66b9a80441781fc83ccf5d82dfa => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:10.552137 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bad60-2251-467d-9f92-1d66924f86ef None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:10.552428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bad60-2251-467d-9f92-1d66924f86ef None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:10.552702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bad60-2251-467d-9f92-1d66924f86ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:10.556937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:10.573339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4c8bad60-2251-467d-9f92-1d66924f86ef None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:10.574578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4c8bad60-2251-467d-9f92-1d66924f86ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:10.633162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1026/2088] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:10.641153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fcbbe-68e6-4cf7-af1b-9115dd05165b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:10.641432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fcbbe-68e6-4cf7-af1b-9115dd05165b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:10.641704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fcbbe-68e6-4cf7-af1b-9115dd05165b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:10.664523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f75fcbbe-68e6-4cf7-af1b-9115dd05165b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:10.665886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f75fcbbe-68e6-4cf7-af1b-9115dd05165b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:10.674119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1063/2089] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:10.678448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bad60-2251-467d-9f92-1d66924f86ef None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:10.688658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bad60-2251-467d-9f92-1d66924f86ef None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:10.694492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67724a-7a44-4d79-b1e4-52eecc2b727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:10.695140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67724a-7a44-4d79-b1e4-52eecc2b727b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:10.695419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67724a-7a44-4d79-b1e4-52eecc2b727b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:10.695698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67724a-7a44-4d79-b1e4-52eecc2b727b admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:10.696346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67724a-7a44-4d79-b1e4-52eecc2b727b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:10.696650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67724a-7a44-4d79-b1e4-52eecc2b727b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:10.698339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67724a-7a44-4d79-b1e4-52eecc2b727b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:10.710607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1027/2090] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:44:10.715340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fcbbe-68e6-4cf7-af1b-9115dd05165b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:10.724665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fcbbe-68e6-4cf7-af1b-9115dd05165b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:10.729421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465d57e-3f1a-4ec4-83f0-fe54e7c58706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:10.730234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d57e-3f1a-4ec4-83f0-fe54e7c58706 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:10.730498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d57e-3f1a-4ec4-83f0-fe54e7c58706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:10.730770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d57e-3f1a-4ec4-83f0-fe54e7c58706 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:10.735844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465d57e-3f1a-4ec4-83f0-fe54e7c58706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:10.735844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465d57e-3f1a-4ec4-83f0-fe54e7c58706 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:10.829581 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1064/2091] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:10 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:10.831532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67724a-7a44-4d79-b1e4-52eecc2b727b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:10.841966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67724a-7a44-4d79-b1e4-52eecc2b727b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:10.917741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa6e50-361d-43ec-a8b8-7450a16ce34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:10.920204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa6e50-361d-43ec-a8b8-7450a16ce34f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:10.920490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa6e50-361d-43ec-a8b8-7450a16ce34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:10.921455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa6e50-361d-43ec-a8b8-7450a16ce34f admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:10.922031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa6e50-361d-43ec-a8b8-7450a16ce34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:10.924580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa6e50-361d-43ec-a8b8-7450a16ce34f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:10.977744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:11.007280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1028/2092] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:10 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:44:11.013260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465d57e-3f1a-4ec4-83f0-fe54e7c58706 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:11.023863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465d57e-3f1a-4ec4-83f0-fe54e7c58706 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:11.028446 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7a256-c77c-4c35-9c11-a6aa8902b6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:11.029076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7a256-c77c-4c35-9c11-a6aa8902b6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.029374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7a256-c77c-4c35-9c11-a6aa8902b6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.029659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f7a256-c77c-4c35-9c11-a6aa8902b6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:11.030339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7a256-c77c-4c35-9c11-a6aa8902b6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:11.031960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f7a256-c77c-4c35-9c11-a6aa8902b6b4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:11.046732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1065/2093] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:11.051402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa6e50-361d-43ec-a8b8-7450a16ce34f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:11.061436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa6e50-361d-43ec-a8b8-7450a16ce34f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:11.067080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c0e21-023e-45bf-af7c-5f3fea3f3525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:11.068062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c0e21-023e-45bf-af7c-5f3fea3f3525 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.068366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c0e21-023e-45bf-af7c-5f3fea3f3525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.068640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c0e21-023e-45bf-af7c-5f3fea3f3525 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:11.069279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c0e21-023e-45bf-af7c-5f3fea3f3525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:11.070888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c0e21-023e-45bf-af7c-5f3fea3f3525 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:11.081476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1029/2094] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:11.085660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7a256-c77c-4c35-9c11-a6aa8902b6b4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:11.096885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f7a256-c77c-4c35-9c11-a6aa8902b6b4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:11.103028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57c879-095b-4bf2-9c49-cc748a2972c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:11.103411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57c879-095b-4bf2-9c49-cc748a2972c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.103698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57c879-095b-4bf2-9c49-cc748a2972c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.103973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57c879-095b-4bf2-9c49-cc748a2972c3 admin admin] PATH_INFO: `/v3/projects/966bae228b294569a3af4d7ea90cf598/users/3122abfc3a6d405bbc2152d5685af189/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:11.123582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57c879-095b-4bf2-9c49-cc748a2972c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966bae228b294569a3af4d7ea90cf598, user_id=3122abfc3a6d405bbc2152d5685af189, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:11.127432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57c879-095b-4bf2-9c49-cc748a2972c3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:11.184551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1066/2095] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:11 2022] PUT /identity/v3/projects/966bae228b294569a3af4d7ea90cf598/users/3122abfc3a6d405bbc2152d5685af189/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:11.195219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3b7e8-5127-47dd-b35f-41434fef9813 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.195504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3b7e8-5127-47dd-b35f-41434fef9813 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.195781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3b7e8-5127-47dd-b35f-41434fef9813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:11.226795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bcc3b7e8-5127-47dd-b35f-41434fef9813 None None] MFA Rules not processed for user `3122abfc3a6d405bbc2152d5685af189`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:11.228143 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bcc3b7e8-5127-47dd-b35f-41434fef9813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:11.292098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1030/2096] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:11.296792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57c879-095b-4bf2-9c49-cc748a2972c3 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:11.360568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:11.364740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57c879-095b-4bf2-9c49-cc748a2972c3 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:11.369402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c5c87-86ac-4a5c-b73a-40c170a01ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:11.370007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c5c87-86ac-4a5c-b73a-40c170a01ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.370329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c5c87-86ac-4a5c-b73a-40c170a01ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.370605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c5c87-86ac-4a5c-b73a-40c170a01ef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:11.371239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c5c87-86ac-4a5c-b73a-40c170a01ef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:11.373185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c5c87-86ac-4a5c-b73a-40c170a01ef5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:11.469968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1067/2097] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:11 2022] POST /identity/v3/projects => generated 369 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:11.474741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3b7e8-5127-47dd-b35f-41434fef9813 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:11.485533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3b7e8-5127-47dd-b35f-41434fef9813 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:11.549386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a6a82-e042-4bb2-8770-41e6ae5afa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:11.549994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a6a82-e042-4bb2-8770-41e6ae5afa64 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.550339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a6a82-e042-4bb2-8770-41e6ae5afa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.550614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a6a82-e042-4bb2-8770-41e6ae5afa64 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:11.551261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a6a82-e042-4bb2-8770-41e6ae5afa64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:11.553474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a6a82-e042-4bb2-8770-41e6ae5afa64 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:11.660679 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1031/2098] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:11 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:11.662528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c5c87-86ac-4a5c-b73a-40c170a01ef5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:11.672932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c5c87-86ac-4a5c-b73a-40c170a01ef5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:11.677791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcc39b-1e09-4bee-8ad5-4c6a1bf052d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:11.678421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc39b-1e09-4bee-8ad5-4c6a1bf052d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.678701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc39b-1e09-4bee-8ad5-4c6a1bf052d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.678973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc39b-1e09-4bee-8ad5-4c6a1bf052d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:11.679650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcc39b-1e09-4bee-8ad5-4c6a1bf052d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:11.681240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcc39b-1e09-4bee-8ad5-4c6a1bf052d9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:11.692920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1068/2099] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:11.697235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a6a82-e042-4bb2-8770-41e6ae5afa64 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:11.707981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a6a82-e042-4bb2-8770-41e6ae5afa64 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:11.712980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff44ed6-818c-4531-9ce2-a8bc254af382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:11.713647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff44ed6-818c-4531-9ce2-a8bc254af382 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.713927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff44ed6-818c-4531-9ce2-a8bc254af382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.714223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff44ed6-818c-4531-9ce2-a8bc254af382 admin admin] PATH_INFO: `/v3/projects/4173e11309354fa591424f2d558afb20/users/327dd18496ef40d7aabeb35c6f9622f6/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:11.729921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff44ed6-818c-4531-9ce2-a8bc254af382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4173e11309354fa591424f2d558afb20, user_id=327dd18496ef40d7aabeb35c6f9622f6, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:11.732888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff44ed6-818c-4531-9ce2-a8bc254af382 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:11.849439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1032/2100] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:11 2022] PUT /identity/v3/projects/4173e11309354fa591424f2d558afb20/users/327dd18496ef40d7aabeb35c6f9622f6/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:11.853273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcc39b-1e09-4bee-8ad5-4c6a1bf052d9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:11.863356 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcc39b-1e09-4bee-8ad5-4c6a1bf052d9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:11.925494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4439a92-8bef-46f8-97e2-77b4d0d42a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:11.926026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4439a92-8bef-46f8-97e2-77b4d0d42a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.926335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4439a92-8bef-46f8-97e2-77b4d0d42a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.926608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4439a92-8bef-46f8-97e2-77b4d0d42a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:11.927247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4439a92-8bef-46f8-97e2-77b4d0d42a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:11.928835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4439a92-8bef-46f8-97e2-77b4d0d42a9a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:11.942689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1069/2101] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:11.943870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff44ed6-818c-4531-9ce2-a8bc254af382 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:11.955902 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff44ed6-818c-4531-9ce2-a8bc254af382 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:11.960604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbdcd7-d1d1-4fa7-b9bb-034a26d804b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:11.961320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbdcd7-d1d1-4fa7-b9bb-034a26d804b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.961658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbdcd7-d1d1-4fa7-b9bb-034a26d804b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.962114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cbdcd7-d1d1-4fa7-b9bb-034a26d804b6 admin admin] PATH_INFO: `/v3/projects/4173e11309354fa591424f2d558afb20/users/327dd18496ef40d7aabeb35c6f9622f6/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:11.964091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cbdcd7-d1d1-4fa7-b9bb-034a26d804b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4173e11309354fa591424f2d558afb20, user_id=327dd18496ef40d7aabeb35c6f9622f6, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:11.966856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cbdcd7-d1d1-4fa7-b9bb-034a26d804b6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:11.982733 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1033/2102] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:11 2022] PUT /identity/v3/projects/4173e11309354fa591424f2d558afb20/users/327dd18496ef40d7aabeb35c6f9622f6/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:11.990908 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ac09-5d0e-4f4e-8e10-3519a5aadb09 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:11.991284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ac09-5d0e-4f4e-8e10-3519a5aadb09 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:11.991687 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ac09-5d0e-4f4e-8e10-3519a5aadb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:12.033142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b998ac09-5d0e-4f4e-8e10-3519a5aadb09 None None] MFA Rules not processed for user `327dd18496ef40d7aabeb35c6f9622f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:12.034443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b998ac09-5d0e-4f4e-8e10-3519a5aadb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:12.084079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:12.152972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1070/2103] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:12.161098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbdcd7-d1d1-4fa7-b9bb-034a26d804b6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:12.162948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4cbdcd7-d1d1-4fa7-b9bb-034a26d804b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:12.230223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cbdcd7-d1d1-4fa7-b9bb-034a26d804b6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:12.234958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222840d0-c9ba-4a88-8fb0-bac5fe54ac15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:12.235580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222840d0-c9ba-4a88-8fb0-bac5fe54ac15 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:12.235909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222840d0-c9ba-4a88-8fb0-bac5fe54ac15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:12.236178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222840d0-c9ba-4a88-8fb0-bac5fe54ac15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:12.246122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222840d0-c9ba-4a88-8fb0-bac5fe54ac15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:12.247591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222840d0-c9ba-4a88-8fb0-bac5fe54ac15 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:12.261755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1034/2104] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:13.065764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:15.815166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998ac09-5d0e-4f4e-8e10-3519a5aadb09 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:15.852494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:15.890128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998ac09-5d0e-4f4e-8e10-3519a5aadb09 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:15.895837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:15.896489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:15.896763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:15.897219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] PATH_INFO: `/v3/users/3122abfc3a6d405bbc2152d5685af189` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:15.898425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3122abfc3a6d405bbc2152d5685af189)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:15.901343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:16.113297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3122abfc3a6d405bbc2152d5685af189'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:16.132551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3122abfc3a6d405bbc2152d5685af189'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:16.133893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3122abfc3a6d405bbc2152d5685af189'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:16.213997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3122abfc3a6d405bbc2152d5685af189'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:16.215837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3122abfc3a6d405bbc2152d5685af189'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:16.228767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1071/2105] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:15 2022] DELETE /identity/v3/users/3122abfc3a6d405bbc2152d5685af189 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:16.233120 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5e5e-632b-47d2-9788-581a53979c72 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:16.233639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5e5e-632b-47d2-9788-581a53979c72 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:16.233744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5e5e-632b-47d2-9788-581a53979c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:16.255943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ef9a5e5e-632b-47d2-9788-581a53979c72 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:16.256650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ef9a5e5e-632b-47d2-9788-581a53979c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:16.331103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1035/2106] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:16.339189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:16.341062 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:16.410234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64cacd-9220-40ca-9aec-3e3747ddc581 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:16.413941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0193da8-6c29-448c-8358-bceb537ab6f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:16.414612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0193da8-6c29-448c-8358-bceb537ab6f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:16.414893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0193da8-6c29-448c-8358-bceb537ab6f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:16.415164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0193da8-6c29-448c-8358-bceb537ab6f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:16.426981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0193da8-6c29-448c-8358-bceb537ab6f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:16.428794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0193da8-6c29-448c-8358-bceb537ab6f3 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:16.446341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1072/2107] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:16.923587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31900152-1521-4cfc-8658-e7a5d3390328 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:16.927271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31900152-1521-4cfc-8658-e7a5d3390328 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:16.927271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31900152-1521-4cfc-8658-e7a5d3390328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:16.958327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-31900152-1521-4cfc-8658-e7a5d3390328 None None] MFA Rules not processed for user `c1b48a973c20470a9af23c77e3da5b72`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:16.959878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-31900152-1521-4cfc-8658-e7a5d3390328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:16.998884 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:17.011018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0193da8-6c29-448c-8358-bceb537ab6f3 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:17.013488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a0193da8-6c29-448c-8358-bceb537ab6f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:17.039487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0193da8-6c29-448c-8358-bceb537ab6f3 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:17.044903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1036/2108] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:17.046389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:17.047334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:17.047752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:17.048171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] PATH_INFO: `/v3/projects/966bae228b294569a3af4d7ea90cf598` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:17.049814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=966bae228b294569a3af4d7ea90cf598)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:17.052759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:17.058267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992f829-3942-40d4-88c7-5e84c5ea1b20 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:17.058718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992f829-3942-40d4-88c7-5e84c5ea1b20 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:17.059092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992f829-3942-40d4-88c7-5e84c5ea1b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:17.093754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f992f829-3942-40d4-88c7-5e84c5ea1b20 None None] MFA Rules not processed for user `c1b48a973c20470a9af23c77e3da5b72`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:17.094092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f992f829-3942-40d4-88c7-5e84c5ea1b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:17.107267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1037/2109] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:17.114828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992f829-3942-40d4-88c7-5e84c5ea1b20 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:17.120043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f992f829-3942-40d4-88c7-5e84c5ea1b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:17.206841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992f829-3942-40d4-88c7-5e84c5ea1b20 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:17.213305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f3b4c-bf79-4c6c-b751-1b68774dfc3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:17.214286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f3b4c-bf79-4c6c-b751-1b68774dfc3c service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:17.214640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f3b4c-bf79-4c6c-b751-1b68774dfc3c service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:17.215011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f3b4c-bf79-4c6c-b751-1b68774dfc3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:17.224252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f3b4c-bf79-4c6c-b751-1b68774dfc3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:17.226807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f3b4c-bf79-4c6c-b751-1b68774dfc3c service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:17.230553 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:17.239054 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966bae228b294569a3af4d7ea90cf598', 'request_id': 'req-001dfe45-5d2d-4195-b997-c05a59129b61'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.239859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1038/2110] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:17.248626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966bae228b294569a3af4d7ea90cf598', 'request_id': 'req-001dfe45-5d2d-4195-b997-c05a59129b61'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.340395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f3b4c-bf79-4c6c-b751-1b68774dfc3c service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:17.342332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-422f3b4c-bf79-4c6c-b751-1b68774dfc3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:17.349069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966bae228b294569a3af4d7ea90cf598', 'request_id': 'req-001dfe45-5d2d-4195-b997-c05a59129b61'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.371190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f3b4c-bf79-4c6c-b751-1b68774dfc3c service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:17.371650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] The token cache is being invalidate because project 966bae228b294569a3af4d7ea90cf598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:17.371946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.374690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.379572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1073/2111] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:17 2022] DELETE /identity/v3/projects/966bae228b294569a3af4d7ea90cf598 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:17.380948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:17.382399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:17.490083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:17.490905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:17.491217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:17.491496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] PATH_INFO: `/v3/users/c18a4de4a5d14a99b4165d55c4a9caf7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:17.492705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18a4de4a5d14a99b4165d55c4a9caf7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:17.494731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:17.551372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001dfe45-5d2d-4195-b997-c05a59129b61 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:17.562433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:17.563257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:17.563543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:17.564114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] PATH_INFO: `/v3/users/327dd18496ef40d7aabeb35c6f9622f6` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:17.567213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327dd18496ef40d7aabeb35c6f9622f6)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:17.569549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:17.851227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327dd18496ef40d7aabeb35c6f9622f6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.857397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18a4de4a5d14a99b4165d55c4a9caf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.858089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18a4de4a5d14a99b4165d55c4a9caf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.868603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:17.874601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327dd18496ef40d7aabeb35c6f9622f6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.876802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327dd18496ef40d7aabeb35c6f9622f6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.887504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18a4de4a5d14a99b4165d55c4a9caf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.962509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327dd18496ef40d7aabeb35c6f9622f6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.963162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ab5f74d8-9ffd-44cb-9d34-caca692ae160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327dd18496ef40d7aabeb35c6f9622f6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:17.978254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1074/2112] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:17 2022] DELETE /identity/v3/users/327dd18496ef40d7aabeb35c6f9622f6 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:18.023287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18a4de4a5d14a99b4165d55c4a9caf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:18.023914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18a4de4a5d14a99b4165d55c4a9caf7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:18.045113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1039/2113] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:17 2022] DELETE /identity/v3/users/c18a4de4a5d14a99b4165d55c4a9caf7 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:18.047869 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3c63-9287-44ee-896f-48cf224162f4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:18.050079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3c63-9287-44ee-896f-48cf224162f4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:18.050543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3c63-9287-44ee-896f-48cf224162f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:18.077801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-476f3c63-9287-44ee-896f-48cf224162f4 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:18.083110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-476f3c63-9287-44ee-896f-48cf224162f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:18.134518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:18.146240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1075/2114] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:18.162688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:18.165481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:18.266180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae47f-990a-482a-9743-1a487bf4b323 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:18.266508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae47f-990a-482a-9743-1a487bf4b323 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:18.266728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae47f-990a-482a-9743-1a487bf4b323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:18.269786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57e123-4532-4ddf-b3c8-e6cbf5dd1e34 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:18.273680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5db0a6-cce1-4430-bc06-e34fc937aea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:18.274527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5db0a6-cce1-4430-bc06-e34fc937aea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:18.274816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5db0a6-cce1-4430-bc06-e34fc937aea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:18.275105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5db0a6-cce1-4430-bc06-e34fc937aea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:18.288393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5db0a6-cce1-4430-bc06-e34fc937aea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:18.291297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5db0a6-cce1-4430-bc06-e34fc937aea4 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:18.295811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5b3ae47f-990a-482a-9743-1a487bf4b323 None None] MFA Rules not processed for user `c1b48a973c20470a9af23c77e3da5b72`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:18.297322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5b3ae47f-990a-482a-9743-1a487bf4b323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:18.305510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1040/2115] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:18.378635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1076/2116] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:18.516315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5db0a6-cce1-4430-bc06-e34fc937aea4 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:18.518454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3a5db0a6-cce1-4430-bc06-e34fc937aea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:18.534188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5db0a6-cce1-4430-bc06-e34fc937aea4 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:18.541066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:18.541864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:18.542203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:18.542438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] PATH_INFO: `/v3/projects/4173e11309354fa591424f2d558afb20` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:18.543782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4173e11309354fa591424f2d558afb20)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:18.545799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:18.719435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4173e11309354fa591424f2d558afb20', 'request_id': 'req-99f4605c-5e4c-4008-a49e-293da7dedb1a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:18.728718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4173e11309354fa591424f2d558afb20', 'request_id': 'req-99f4605c-5e4c-4008-a49e-293da7dedb1a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:18.838426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4173e11309354fa591424f2d558afb20', 'request_id': 'req-99f4605c-5e4c-4008-a49e-293da7dedb1a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:18.867522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] The token cache is being invalidate because project 4173e11309354fa591424f2d558afb20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:18.867837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:18.869096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-99f4605c-5e4c-4008-a49e-293da7dedb1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:18.871465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1041/2117] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:18 2022] DELETE /identity/v3/projects/4173e11309354fa591424f2d558afb20 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:18.876425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b71f-b1ce-42ef-9917-66cdc592ca98 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:18.877249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b71f-b1ce-42ef-9917-66cdc592ca98 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:18.877514 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b71f-b1ce-42ef-9917-66cdc592ca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:18.901497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d771b71f-b1ce-42ef-9917-66cdc592ca98 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:18.901497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d771b71f-b1ce-42ef-9917-66cdc592ca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:18.979320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1077/2118] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:18.988691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8324459-c772-46f2-bc2c-8083ea3907d8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:18.988992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8324459-c772-46f2-bc2c-8083ea3907d8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:18.991448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8324459-c772-46f2-bc2c-8083ea3907d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:19.014963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c8324459-c772-46f2-bc2c-8083ea3907d8 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:19.023929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c8324459-c772-46f2-bc2c-8083ea3907d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:19.032300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1042/2119] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:19.039579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8324459-c772-46f2-bc2c-8083ea3907d8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:19.039811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b71f-b1ce-42ef-9917-66cdc592ca98 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:19.042336 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c8324459-c772-46f2-bc2c-8083ea3907d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:19.054899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b71f-b1ce-42ef-9917-66cdc592ca98 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:19.062686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8324459-c772-46f2-bc2c-8083ea3907d8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:19.068320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:19.068961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:19.069286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:19.069602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] PATH_INFO: `/v3/projects/7ca58828350643f8a8194369351c590d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:19.070103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b09fb-577d-4565-916c-018651913aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:19.070755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b09fb-577d-4565-916c-018651913aad admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:19.070885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca58828350643f8a8194369351c590d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:19.071182 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b09fb-577d-4565-916c-018651913aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:19.071461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b09fb-577d-4565-916c-018651913aad admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:19.072120 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b09fb-577d-4565-916c-018651913aad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:19.072427 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b09fb-577d-4565-916c-018651913aad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:19.072757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:19.074032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b09fb-577d-4565-916c-018651913aad admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:19.107056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1078/2120] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:44:19.108069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b09fb-577d-4565-916c-018651913aad admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:19.131842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b09fb-577d-4565-916c-018651913aad admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:19.137103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e5b09-6b93-44ed-8133-aa996c5be346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:19.138091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e5b09-6b93-44ed-8133-aa996c5be346 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:19.138707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e5b09-6b93-44ed-8133-aa996c5be346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:19.139256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e5b09-6b93-44ed-8133-aa996c5be346 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:19.139893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e5b09-6b93-44ed-8133-aa996c5be346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:19.141950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e5b09-6b93-44ed-8133-aa996c5be346 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:19.313490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1079/2121] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:19 2022] POST /identity/v3/projects => generated 369 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:19.318843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e5b09-6b93-44ed-8133-aa996c5be346 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:19.325792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:19.330006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca58828350643f8a8194369351c590d', 'request_id': 'req-0cffede0-bf7b-4989-990c-6972ee266a3f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:19.338355 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca58828350643f8a8194369351c590d', 'request_id': 'req-0cffede0-bf7b-4989-990c-6972ee266a3f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:19.454264 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca58828350643f8a8194369351c590d', 'request_id': 'req-0cffede0-bf7b-4989-990c-6972ee266a3f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:19.468448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] The token cache is being invalidate because project 7ca58828350643f8a8194369351c590d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:19.468751 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:19.469789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:19.474234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1043/2122] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:19 2022] DELETE /identity/v3/projects/7ca58828350643f8a8194369351c590d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:19.478443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:19.483141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:19.489603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e5b09-6b93-44ed-8133-aa996c5be346 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:19.510344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c5bd4-de9c-4d9f-8608-8dedc0bbe419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:19.510344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c5bd4-de9c-4d9f-8608-8dedc0bbe419 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:19.510344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c5bd4-de9c-4d9f-8608-8dedc0bbe419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:19.510344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c5bd4-de9c-4d9f-8608-8dedc0bbe419 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:19.510690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c5bd4-de9c-4d9f-8608-8dedc0bbe419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:19.510690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c5bd4-de9c-4d9f-8608-8dedc0bbe419 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:19.533370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:19.603487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffede0-bf7b-4989-990c-6972ee266a3f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:19.609494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:19.609494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:19.609856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:19.610174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] PATH_INFO: `/v3/users/9fe9f43219144ec2af347356ed2f8b58` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:19.613066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe9f43219144ec2af347356ed2f8b58)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:19.616038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:19.638704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1080/2123] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:19 2022] POST /identity/v3/users => generated 285 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:44:19.642897 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c5bd4-de9c-4d9f-8608-8dedc0bbe419 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:19.647055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-572c5bd4-de9c-4d9f-8608-8dedc0bbe419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:19.668635 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c5bd4-de9c-4d9f-8608-8dedc0bbe419 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:19.675451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b38bd-e219-49e2-8fc5-1fa2828ada8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:19.676065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b38bd-e219-49e2-8fc5-1fa2828ada8e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:19.676342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b38bd-e219-49e2-8fc5-1fa2828ada8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:19.676613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b38bd-e219-49e2-8fc5-1fa2828ada8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:19.682456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b38bd-e219-49e2-8fc5-1fa2828ada8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:19.694219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b38bd-e219-49e2-8fc5-1fa2828ada8e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:19.706118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1081/2124] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:44:19.710126 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b38bd-e219-49e2-8fc5-1fa2828ada8e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:19.720195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b38bd-e219-49e2-8fc5-1fa2828ada8e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:19.725403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878e5a2-4ea7-4aff-a66e-4eeb47900753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:19.725994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878e5a2-4ea7-4aff-a66e-4eeb47900753 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:19.726320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878e5a2-4ea7-4aff-a66e-4eeb47900753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:19.726589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878e5a2-4ea7-4aff-a66e-4eeb47900753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:19.727218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878e5a2-4ea7-4aff-a66e-4eeb47900753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:19.728873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878e5a2-4ea7-4aff-a66e-4eeb47900753 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:19.742170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1082/2125] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:19.745165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878e5a2-4ea7-4aff-a66e-4eeb47900753 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:19.754721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878e5a2-4ea7-4aff-a66e-4eeb47900753 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:19.759629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81687e06-bf77-4c84-b567-68ca3ae9fb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:19.760564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81687e06-bf77-4c84-b567-68ca3ae9fb39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:19.761125 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81687e06-bf77-4c84-b567-68ca3ae9fb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:19.761558 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81687e06-bf77-4c84-b567-68ca3ae9fb39 admin admin] PATH_INFO: `/v3/projects/2feaea4adb794093805a08f3a81cb33b/users/ef32a452ae634b6289ca55bb2b1016af/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:19.785741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81687e06-bf77-4c84-b567-68ca3ae9fb39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2feaea4adb794093805a08f3a81cb33b, user_id=ef32a452ae634b6289ca55bb2b1016af, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:19.788854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81687e06-bf77-4c84-b567-68ca3ae9fb39 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:19.921398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe9f43219144ec2af347356ed2f8b58'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:19.923969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe9f43219144ec2af347356ed2f8b58'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:19.930800 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1083/2126] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:19 2022] PUT /identity/v3/projects/2feaea4adb794093805a08f3a81cb33b/users/ef32a452ae634b6289ca55bb2b1016af/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:19.943417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcedc8-0132-4b3a-b5df-76f840f7bf37 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:19.943982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcedc8-0132-4b3a-b5df-76f840f7bf37 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:19.944370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcedc8-0132-4b3a-b5df-76f840f7bf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:19.961773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe9f43219144ec2af347356ed2f8b58'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:19.989223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f3dcedc8-0132-4b3a-b5df-76f840f7bf37 None None] MFA Rules not processed for user `ef32a452ae634b6289ca55bb2b1016af`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:19.989223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f3dcedc8-0132-4b3a-b5df-76f840f7bf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:20.063071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1084/2127] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:20.067415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe9f43219144ec2af347356ed2f8b58'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:20.068298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe9f43219144ec2af347356ed2f8b58'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:20.082693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcedc8-0132-4b3a-b5df-76f840f7bf37 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:20.084903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f3dcedc8-0132-4b3a-b5df-76f840f7bf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:20.101298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1044/2128] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:19 2022] DELETE /identity/v3/users/9fe9f43219144ec2af347356ed2f8b58 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:20.163638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcedc8-0132-4b3a-b5df-76f840f7bf37 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:20.256896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108ebc-5cbc-4a3f-b3d9-98e267430ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:20.262878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108ebc-5cbc-4a3f-b3d9-98e267430ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:20.262878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108ebc-5cbc-4a3f-b3d9-98e267430ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:20.262878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108ebc-5cbc-4a3f-b3d9-98e267430ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:20.262878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17108ebc-5cbc-4a3f-b3d9-98e267430ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:20.262878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17108ebc-5cbc-4a3f-b3d9-98e267430ef7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:20.408856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:20.418551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1085/2129] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:20 2022] POST /identity/v3/projects => generated 367 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:20.421386 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:20.439544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ef2f9-8eb0-4717-b8d4-e222e2b50b50 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:20.549210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7ed30-577d-413b-ad4f-3035a02e7015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:20.549950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7ed30-577d-413b-ad4f-3035a02e7015 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:20.550356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7ed30-577d-413b-ad4f-3035a02e7015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:20.551787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7ed30-577d-413b-ad4f-3035a02e7015 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:20.552532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7ed30-577d-413b-ad4f-3035a02e7015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:20.554588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7ed30-577d-413b-ad4f-3035a02e7015 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:20.692036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1045/2130] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:20 2022] POST /identity/v3/users => generated 290 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:20.696085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108ebc-5cbc-4a3f-b3d9-98e267430ef7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:20.712272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:20.722018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108ebc-5cbc-4a3f-b3d9-98e267430ef7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:20.728430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7ed30-577d-413b-ad4f-3035a02e7015 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:20.731951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9fb7ed30-577d-413b-ad4f-3035a02e7015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:20.740405 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4cdf9-13d9-4f68-9f08-9b8d40c6e9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:20.741294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4cdf9-13d9-4f68-9f08-9b8d40c6e9ea admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:20.741730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4cdf9-13d9-4f68-9f08-9b8d40c6e9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:20.742170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4cdf9-13d9-4f68-9f08-9b8d40c6e9ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:20.742993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4cdf9-13d9-4f68-9f08-9b8d40c6e9ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:20.744700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4cdf9-13d9-4f68-9f08-9b8d40c6e9ea admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:20.754011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7ed30-577d-413b-ad4f-3035a02e7015 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:20.762260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1086/2131] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:20.766731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4cdf9-13d9-4f68-9f08-9b8d40c6e9ea admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:20.769442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:20.770194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:20.770438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:20.770716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] PATH_INFO: `/v3/projects/15e70527aeb14f6bbe146768f91e3ff2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:20.774630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e70527aeb14f6bbe146768f91e3ff2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:20.776570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:20.779288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4cdf9-13d9-4f68-9f08-9b8d40c6e9ea admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:20.790748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b19e22-d701-4764-abbf-4530e6fb6ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:20.791436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b19e22-d701-4764-abbf-4530e6fb6ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:20.791820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b19e22-d701-4764-abbf-4530e6fb6ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:20.792596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b19e22-d701-4764-abbf-4530e6fb6ea9 admin admin] PATH_INFO: `/v3/projects/1542e6357c024d14ba1b5e830152ba67/users/f4e22eb9e73e4a169ae63a50978d2beb/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:20.866691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b19e22-d701-4764-abbf-4530e6fb6ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1542e6357c024d14ba1b5e830152ba67, user_id=f4e22eb9e73e4a169ae63a50978d2beb, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:20.870954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b19e22-d701-4764-abbf-4530e6fb6ea9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:21.021562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1087/2132] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:20 2022] PUT /identity/v3/projects/1542e6357c024d14ba1b5e830152ba67/users/f4e22eb9e73e4a169ae63a50978d2beb/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:21.025568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b19e22-d701-4764-abbf-4530e6fb6ea9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:21.036270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e70527aeb14f6bbe146768f91e3ff2', 'request_id': 'req-c60565c2-67fe-4177-a449-aa4d0fea883d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:21.045484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e70527aeb14f6bbe146768f91e3ff2', 'request_id': 'req-c60565c2-67fe-4177-a449-aa4d0fea883d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:21.118886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b19e22-d701-4764-abbf-4530e6fb6ea9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:21.122240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e70527aeb14f6bbe146768f91e3ff2', 'request_id': 'req-c60565c2-67fe-4177-a449-aa4d0fea883d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:21.137445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] The token cache is being invalidate because project 15e70527aeb14f6bbe146768f91e3ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:21.138340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:21.139003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c60565c2-67fe-4177-a449-aa4d0fea883d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:21.142300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1046/2133] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:20 2022] DELETE /identity/v3/projects/15e70527aeb14f6bbe146768f91e3ff2 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:21.148633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9064ae9-34bb-4ce8-995e-ec7e8102ac8d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.148921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9064ae9-34bb-4ce8-995e-ec7e8102ac8d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.149199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9064ae9-34bb-4ce8-995e-ec7e8102ac8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.176630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b9064ae9-34bb-4ce8-995e-ec7e8102ac8d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:21.177805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b9064ae9-34bb-4ce8-995e-ec7e8102ac8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:21.212347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d59a7-77c5-4fcb-a9b5-131d3bbb9828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:21.213004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d59a7-77c5-4fcb-a9b5-131d3bbb9828 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.213284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d59a7-77c5-4fcb-a9b5-131d3bbb9828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.213554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d59a7-77c5-4fcb-a9b5-131d3bbb9828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.214240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d59a7-77c5-4fcb-a9b5-131d3bbb9828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:21.216046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d59a7-77c5-4fcb-a9b5-131d3bbb9828 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:21.228439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1088/2134] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:21.235178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d59a7-77c5-4fcb-a9b5-131d3bbb9828 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:21.237190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-293d59a7-77c5-4fcb-a9b5-131d3bbb9828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:21.250697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d59a7-77c5-4fcb-a9b5-131d3bbb9828 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:21.256196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd227470-dc15-4e1f-8529-fef7b08e84f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:21.256984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd227470-dc15-4e1f-8529-fef7b08e84f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.257259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd227470-dc15-4e1f-8529-fef7b08e84f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.257628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd227470-dc15-4e1f-8529-fef7b08e84f2 admin admin] PATH_INFO: `/v3/projects/1542e6357c024d14ba1b5e830152ba67/users/f4e22eb9e73e4a169ae63a50978d2beb/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.259262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1047/2135] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:21.268069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c977a63-a315-402e-aaaf-27d6368d5931 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.268394 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd227470-dc15-4e1f-8529-fef7b08e84f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1542e6357c024d14ba1b5e830152ba67, user_id=f4e22eb9e73e4a169ae63a50978d2beb, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:21.268540 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c977a63-a315-402e-aaaf-27d6368d5931 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.268719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c977a63-a315-402e-aaaf-27d6368d5931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.271257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd227470-dc15-4e1f-8529-fef7b08e84f2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:21.285997 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1089/2136] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:21 2022] PUT /identity/v3/projects/1542e6357c024d14ba1b5e830152ba67/users/f4e22eb9e73e4a169ae63a50978d2beb/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:21.291227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8c977a63-a315-402e-aaaf-27d6368d5931 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:21.292052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8c977a63-a315-402e-aaaf-27d6368d5931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:21.296709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124e0ad-35a4-4165-9be9-cca6044915c6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.297756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124e0ad-35a4-4165-9be9-cca6044915c6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.299035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124e0ad-35a4-4165-9be9-cca6044915c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.342172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a124e0ad-35a4-4165-9be9-cca6044915c6 None None] MFA Rules not processed for user `f4e22eb9e73e4a169ae63a50978d2beb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:21.344408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a124e0ad-35a4-4165-9be9-cca6044915c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:21.365513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:21.389812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1048/2137] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:21.394674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c977a63-a315-402e-aaaf-27d6368d5931 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:21.410115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c977a63-a315-402e-aaaf-27d6368d5931 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:21.420742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398565b6-9f70-4026-9aa8-eb1d10d0f1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:21.426103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398565b6-9f70-4026-9aa8-eb1d10d0f1aa admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.426103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398565b6-9f70-4026-9aa8-eb1d10d0f1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.426103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398565b6-9f70-4026-9aa8-eb1d10d0f1aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.426103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398565b6-9f70-4026-9aa8-eb1d10d0f1aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:21.426103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398565b6-9f70-4026-9aa8-eb1d10d0f1aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:21.426103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398565b6-9f70-4026-9aa8-eb1d10d0f1aa admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:21.436456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1090/2138] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:21.442209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1049/2139] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:44:21.446267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124e0ad-35a4-4165-9be9-cca6044915c6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:21.455742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398565b6-9f70-4026-9aa8-eb1d10d0f1aa admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:21.457697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-398565b6-9f70-4026-9aa8-eb1d10d0f1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:21.458815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124e0ad-35a4-4165-9be9-cca6044915c6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:21.464211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d16063-273e-40fa-a7bc-d49c38f78258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:21.465050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d16063-273e-40fa-a7bc-d49c38f78258 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.465528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d16063-273e-40fa-a7bc-d49c38f78258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.465987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d16063-273e-40fa-a7bc-d49c38f78258 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.467148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d16063-273e-40fa-a7bc-d49c38f78258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:21.469531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d16063-273e-40fa-a7bc-d49c38f78258 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:21.569943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1091/2140] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:21 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:21.576321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d16063-273e-40fa-a7bc-d49c38f78258 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:21.632763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:21.686435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398565b6-9f70-4026-9aa8-eb1d10d0f1aa admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:21.702064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d16063-273e-40fa-a7bc-d49c38f78258 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:21.709035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76ca45-97a1-441d-b164-7f1bff69bd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:21.710130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76ca45-97a1-441d-b164-7f1bff69bd2b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.710705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76ca45-97a1-441d-b164-7f1bff69bd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.711062 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76ca45-97a1-441d-b164-7f1bff69bd2b admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.711703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76ca45-97a1-441d-b164-7f1bff69bd2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:21.713613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76ca45-97a1-441d-b164-7f1bff69bd2b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:21.761739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32a208-447c-4a98-9216-d6bbfcf61418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:21.761739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32a208-447c-4a98-9216-d6bbfcf61418 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.761739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32a208-447c-4a98-9216-d6bbfcf61418 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.761739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32a208-447c-4a98-9216-d6bbfcf61418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.854401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1092/2141] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:21 2022] POST /identity/v3/users => generated 286 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:21.856462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76ca45-97a1-441d-b164-7f1bff69bd2b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:21.870022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32a208-447c-4a98-9216-d6bbfcf61418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:21.872107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32a208-447c-4a98-9216-d6bbfcf61418 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:21.883070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76ca45-97a1-441d-b164-7f1bff69bd2b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:21.886848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1050/2142] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:21.887954 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1fa4-4331-40b3-a3b9-f2f715003c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:21.888565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1fa4-4331-40b3-a3b9-f2f715003c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.888862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1fa4-4331-40b3-a3b9-f2f715003c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.893681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1fa4-4331-40b3-a3b9-f2f715003c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.893681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd1fa4-4331-40b3-a3b9-f2f715003c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:21.893681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd1fa4-4331-40b3-a3b9-f2f715003c63 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:21.924234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1093/2143] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:21.928674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32a208-447c-4a98-9216-d6bbfcf61418 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:21.947502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32a208-447c-4a98-9216-d6bbfcf61418 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:21.952801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743c5e7-8c43-41f4-abda-80d0b35e16db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:21.953482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743c5e7-8c43-41f4-abda-80d0b35e16db admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:21.953833 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743c5e7-8c43-41f4-abda-80d0b35e16db admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:21.954201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743c5e7-8c43-41f4-abda-80d0b35e16db admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:21.955498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743c5e7-8c43-41f4-abda-80d0b35e16db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:21.958949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743c5e7-8c43-41f4-abda-80d0b35e16db admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:21.977710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1051/2144] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:21.981698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1fa4-4331-40b3-a3b9-f2f715003c63 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:21.993729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1fa4-4331-40b3-a3b9-f2f715003c63 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:21.999528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f965f-662d-4396-a4bc-de7d32e03e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:22.000393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f965f-662d-4396-a4bc-de7d32e03e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:22.000700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f965f-662d-4396-a4bc-de7d32e03e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:22.001011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f965f-662d-4396-a4bc-de7d32e03e45 admin admin] PATH_INFO: `/v3/projects/02e256441535490eb58653201bc7e0e7/users/b16e9fb7b73a45979dd52e4c48dbdae7/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:22.019933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f965f-662d-4396-a4bc-de7d32e03e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e256441535490eb58653201bc7e0e7, user_id=b16e9fb7b73a45979dd52e4c48dbdae7, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:22.023149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f965f-662d-4396-a4bc-de7d32e03e45 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:22.083192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743c5e7-8c43-41f4-abda-80d0b35e16db admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:22.098054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743c5e7-8c43-41f4-abda-80d0b35e16db admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:22.101086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1094/2145] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:21 2022] PUT /identity/v3/projects/02e256441535490eb58653201bc7e0e7/users/b16e9fb7b73a45979dd52e4c48dbdae7/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:22.111194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907cc92-3290-4bce-96f0-b28910ef0058 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:22.111480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907cc92-3290-4bce-96f0-b28910ef0058 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:22.111976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907cc92-3290-4bce-96f0-b28910ef0058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:22.145322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5907cc92-3290-4bce-96f0-b28910ef0058 None None] MFA Rules not processed for user `b16e9fb7b73a45979dd52e4c48dbdae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:22.146599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5907cc92-3290-4bce-96f0-b28910ef0058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:22.149417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:22.154434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2240a-d7a2-439b-ae88-4acb86b29a80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:22.155180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2240a-d7a2-439b-ae88-4acb86b29a80 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:22.155461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2240a-d7a2-439b-ae88-4acb86b29a80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:22.159736 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2240a-d7a2-439b-ae88-4acb86b29a80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:22.159736 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-95b2240a-d7a2-439b-ae88-4acb86b29a80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:22.184570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:22.257094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1095/2146] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:22.257498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2240a-d7a2-439b-ae88-4acb86b29a80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:22.257498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2240a-d7a2-439b-ae88-4acb86b29a80 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:22.257498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1052/2147] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:44:22.654654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907cc92-3290-4bce-96f0-b28910ef0058 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:22.656888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5907cc92-3290-4bce-96f0-b28910ef0058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:22.668824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907cc92-3290-4bce-96f0-b28910ef0058 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:22.672446 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7f9e0-6032-4cdd-9444-3d847838ad99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:22.673052 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7f9e0-6032-4cdd-9444-3d847838ad99 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:22.673333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7f9e0-6032-4cdd-9444-3d847838ad99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:22.673609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7f9e0-6032-4cdd-9444-3d847838ad99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:22.682647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7f9e0-6032-4cdd-9444-3d847838ad99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:22.688259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7f9e0-6032-4cdd-9444-3d847838ad99 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:22.697044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1096/2148] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:22.718905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2240a-d7a2-439b-ae88-4acb86b29a80 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:22.804689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2240a-d7a2-439b-ae88-4acb86b29a80 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:22.815192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:22.816069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:22.816424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:22.816694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] PATH_INFO: `/v3/users/b16e9fb7b73a45979dd52e4c48dbdae7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:22.817904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b16e9fb7b73a45979dd52e4c48dbdae7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:22.819891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:23.081450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b16e9fb7b73a45979dd52e4c48dbdae7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:23.081450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b16e9fb7b73a45979dd52e4c48dbdae7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:23.108782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b16e9fb7b73a45979dd52e4c48dbdae7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:23.174388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b16e9fb7b73a45979dd52e4c48dbdae7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:23.175019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b16e9fb7b73a45979dd52e4c48dbdae7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:23.195904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1053/2149] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:22 2022] DELETE /identity/v3/users/b16e9fb7b73a45979dd52e4c48dbdae7 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:23.200546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec2227-e3ee-4ca4-9c59-6da06ae1fc55 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:23.201016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec2227-e3ee-4ca4-9c59-6da06ae1fc55 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:23.201306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec2227-e3ee-4ca4-9c59-6da06ae1fc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:23.221650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-aeec2227-e3ee-4ca4-9c59-6da06ae1fc55 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:23.223347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aeec2227-e3ee-4ca4-9c59-6da06ae1fc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:23.289051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1097/2150] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:23.303222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:23.308257 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:23.335476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:23.375873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfef31-9784-4c8a-bb29-19de397e3354 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:23.395651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc8ec9-e5c3-4d87-aa82-364ef89838dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:23.396282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc8ec9-e5c3-4d87-aa82-364ef89838dc service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:23.396700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc8ec9-e5c3-4d87-aa82-364ef89838dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:23.396980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc8ec9-e5c3-4d87-aa82-364ef89838dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:23.426510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc8ec9-e5c3-4d87-aa82-364ef89838dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:23.440621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc8ec9-e5c3-4d87-aa82-364ef89838dc service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:23.462910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1054/2151] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:24.255795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec2227-e3ee-4ca4-9c59-6da06ae1fc55 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:24.259297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aeec2227-e3ee-4ca4-9c59-6da06ae1fc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:24.287900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec2227-e3ee-4ca4-9c59-6da06ae1fc55 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:24.293466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:24.294231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:24.294516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:24.294793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] PATH_INFO: `/v3/projects/02e256441535490eb58653201bc7e0e7` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:24.295978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02e256441535490eb58653201bc7e0e7)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:24.297920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:24.439598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02e256441535490eb58653201bc7e0e7', 'request_id': 'req-16ad2a0b-923d-4505-92e4-7b65dbc82663'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:24.457241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02e256441535490eb58653201bc7e0e7', 'request_id': 'req-16ad2a0b-923d-4505-92e4-7b65dbc82663'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:24.531231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02e256441535490eb58653201bc7e0e7', 'request_id': 'req-16ad2a0b-923d-4505-92e4-7b65dbc82663'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:24.543965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] The token cache is being invalidate because project 02e256441535490eb58653201bc7e0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:24.544190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:24.544832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-16ad2a0b-923d-4505-92e4-7b65dbc82663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:24.546374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1098/2152] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:24 2022] DELETE /identity/v3/projects/02e256441535490eb58653201bc7e0e7 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:24.551476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70e7e-1ac4-45fb-8134-bb7ab084ec09 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:24.551966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70e7e-1ac4-45fb-8134-bb7ab084ec09 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:24.552070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70e7e-1ac4-45fb-8134-bb7ab084ec09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:24.581633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b4f70e7e-1ac4-45fb-8134-bb7ab084ec09 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:24.592282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b4f70e7e-1ac4-45fb-8134-bb7ab084ec09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:24.729570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1055/2153] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:24.750690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cad6a-e240-4e59-a9a3-66525626fd48 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:24.752020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cad6a-e240-4e59-a9a3-66525626fd48 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:24.752479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cad6a-e240-4e59-a9a3-66525626fd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:24.797550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-830cad6a-e240-4e59-a9a3-66525626fd48 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:24.798896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-830cad6a-e240-4e59-a9a3-66525626fd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:24.820639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1099/2154] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:24.825739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f70e7e-1ac4-45fb-8134-bb7ab084ec09 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:24.849561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f70e7e-1ac4-45fb-8134-bb7ab084ec09 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:24.860709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77537682-d696-4dbc-8500-33cc46378bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:24.861628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77537682-d696-4dbc-8500-33cc46378bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:24.861864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77537682-d696-4dbc-8500-33cc46378bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:24.862245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77537682-d696-4dbc-8500-33cc46378bb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:24.862989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77537682-d696-4dbc-8500-33cc46378bb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:24.863485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77537682-d696-4dbc-8500-33cc46378bb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:24.865113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77537682-d696-4dbc-8500-33cc46378bb0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:24.878432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1056/2155] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:44:24.883767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cad6a-e240-4e59-a9a3-66525626fd48 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:24.905374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cad6a-e240-4e59-a9a3-66525626fd48 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:24.911974 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a608b-a73c-4291-8fc7-6ee9da13058d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:24.912710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a608b-a73c-4291-8fc7-6ee9da13058d admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:24.913023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a608b-a73c-4291-8fc7-6ee9da13058d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:24.913338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a608b-a73c-4291-8fc7-6ee9da13058d admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:24.914730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a608b-a73c-4291-8fc7-6ee9da13058d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:24.917230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a608b-a73c-4291-8fc7-6ee9da13058d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:25.025754 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1100/2156] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:24 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:25.029768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77537682-d696-4dbc-8500-33cc46378bb0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:25.043148 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77537682-d696-4dbc-8500-33cc46378bb0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:25.201927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b65c92-61b6-423b-b943-d6df5370e954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:25.202462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b65c92-61b6-423b-b943-d6df5370e954 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:25.202888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b65c92-61b6-423b-b943-d6df5370e954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:25.203785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b65c92-61b6-423b-b943-d6df5370e954 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:25.204453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b65c92-61b6-423b-b943-d6df5370e954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:25.206544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b65c92-61b6-423b-b943-d6df5370e954 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:25.341426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1057/2157] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:25 2022] POST /identity/v3/users => generated 289 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:25.346112 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a608b-a73c-4291-8fc7-6ee9da13058d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:25.371412 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a608b-a73c-4291-8fc7-6ee9da13058d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:25.381812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec3f8e-087d-4baf-be71-99c6179003c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:25.382417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec3f8e-087d-4baf-be71-99c6179003c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:25.382730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec3f8e-087d-4baf-be71-99c6179003c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:25.383280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec3f8e-087d-4baf-be71-99c6179003c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:25.384138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec3f8e-087d-4baf-be71-99c6179003c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:25.385730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec3f8e-087d-4baf-be71-99c6179003c4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:25.412361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1101/2158] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:25.415982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b65c92-61b6-423b-b943-d6df5370e954 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:25.433475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b65c92-61b6-423b-b943-d6df5370e954 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:25.473728 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eccb0a-ee2f-4409-8bd9-6b59aac32bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:25.474389 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eccb0a-ee2f-4409-8bd9-6b59aac32bbe admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:25.474676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eccb0a-ee2f-4409-8bd9-6b59aac32bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:25.474945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eccb0a-ee2f-4409-8bd9-6b59aac32bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:25.475634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eccb0a-ee2f-4409-8bd9-6b59aac32bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:25.477209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eccb0a-ee2f-4409-8bd9-6b59aac32bbe admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:25.493470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1058/2159] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:25.498188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec3f8e-087d-4baf-be71-99c6179003c4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:25.503481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:25.508956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec3f8e-087d-4baf-be71-99c6179003c4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:25.513599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1bcee-4c6b-466f-839b-9e9255ec0990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:25.514358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1bcee-4c6b-466f-839b-9e9255ec0990 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:25.514641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1bcee-4c6b-466f-839b-9e9255ec0990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:25.514906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1bcee-4c6b-466f-839b-9e9255ec0990 admin admin] PATH_INFO: `/v3/projects/3ff186fb007547e2b12bd93a3801f6f1/users/4da5cc3812294213b4a194173b521205/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:25.530543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1bcee-4c6b-466f-839b-9e9255ec0990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff186fb007547e2b12bd93a3801f6f1, user_id=4da5cc3812294213b4a194173b521205, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:25.533769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1bcee-4c6b-466f-839b-9e9255ec0990 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:25.645996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1102/2160] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:25 2022] PUT /identity/v3/projects/3ff186fb007547e2b12bd93a3801f6f1/users/4da5cc3812294213b4a194173b521205/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:25.657012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9d024-90c9-4f59-a6d4-5dec2a61446c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:25.658049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9d024-90c9-4f59-a6d4-5dec2a61446c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:25.658457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9d024-90c9-4f59-a6d4-5dec2a61446c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:25.706381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-faf9d024-90c9-4f59-a6d4-5dec2a61446c None None] MFA Rules not processed for user `4da5cc3812294213b4a194173b521205`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:25.707629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-faf9d024-90c9-4f59-a6d4-5dec2a61446c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:25.731331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:25.781576 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1059/2161] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:25.797699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1bcee-4c6b-466f-839b-9e9255ec0990 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:25.800093 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aaa1bcee-4c6b-466f-839b-9e9255ec0990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:25.870337 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1bcee-4c6b-466f-839b-9e9255ec0990 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:25.878292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d01ff3-fc98-4f01-a963-ba1ff89bc3cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:25.878292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d01ff3-fc98-4f01-a963-ba1ff89bc3cd service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:25.878292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d01ff3-fc98-4f01-a963-ba1ff89bc3cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:25.878292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d01ff3-fc98-4f01-a963-ba1ff89bc3cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:25.904578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d01ff3-fc98-4f01-a963-ba1ff89bc3cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:25.907031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d01ff3-fc98-4f01-a963-ba1ff89bc3cd service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:25.917281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1103/2162] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:44:27.362242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6295c-b39d-4e17-83b0-03c760044543 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:27.362781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6295c-b39d-4e17-83b0-03c760044543 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:27.363014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6295c-b39d-4e17-83b0-03c760044543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:27.383465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ade6295c-b39d-4e17-83b0-03c760044543 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:27.389664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ade6295c-b39d-4e17-83b0-03c760044543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:27.456742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1060/2163] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:27.529503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d01ff3-fc98-4f01-a963-ba1ff89bc3cd service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:27.531229 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a7d01ff3-fc98-4f01-a963-ba1ff89bc3cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:27.670675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d01ff3-fc98-4f01-a963-ba1ff89bc3cd service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:27.677164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55d4b7-ee82-4195-992d-d67530435180 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:27.678051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55d4b7-ee82-4195-992d-d67530435180 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:27.678750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55d4b7-ee82-4195-992d-d67530435180 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:27.679080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55d4b7-ee82-4195-992d-d67530435180 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:27.688490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55d4b7-ee82-4195-992d-d67530435180 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:27.690764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55d4b7-ee82-4195-992d-d67530435180 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:27.700761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1104/2164] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:44:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:44:28.739227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719efe4-9479-46f3-8346-ba0a2f860f94 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:28.739728 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719efe4-9479-46f3-8346-ba0a2f860f94 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:28.739818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719efe4-9479-46f3-8346-ba0a2f860f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:28.760246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4719efe4-9479-46f3-8346-ba0a2f860f94 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:28.761894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4719efe4-9479-46f3-8346-ba0a2f860f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:28.769915 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1061/2165] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 05:44:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:28.783138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55d4b7-ee82-4195-992d-d67530435180 service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:28.794098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55d4b7-ee82-4195-992d-d67530435180 service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:28.800383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e5cbc-7c5f-422a-a694-aec8c0cdb51b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:28.810176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5cbc-7c5f-422a-a694-aec8c0cdb51b service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:28.810489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5cbc-7c5f-422a-a694-aec8c0cdb51b service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:28.810770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5cbc-7c5f-422a-a694-aec8c0cdb51b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:28.820912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e5cbc-7c5f-422a-a694-aec8c0cdb51b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:28.822692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e5cbc-7c5f-422a-a694-aec8c0cdb51b service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:28.834698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1105/2166] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:44:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:44:28.860261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719efe4-9479-46f3-8346-ba0a2f860f94 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:28.862430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4719efe4-9479-46f3-8346-ba0a2f860f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:28.950327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719efe4-9479-46f3-8346-ba0a2f860f94 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:28.955345 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:28.956039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:28.956243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:28.956518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] PATH_INFO: `/v3/users/ef32a452ae634b6289ca55bb2b1016af` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:28.957677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef32a452ae634b6289ca55bb2b1016af)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:28.959611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:29.222660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef32a452ae634b6289ca55bb2b1016af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:29.223534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef32a452ae634b6289ca55bb2b1016af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:29.233705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:29.236313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef32a452ae634b6289ca55bb2b1016af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:29.379447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef32a452ae634b6289ca55bb2b1016af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:29.380209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef32a452ae634b6289ca55bb2b1016af'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:29.395668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1062/2167] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:28 2022] DELETE /identity/v3/users/ef32a452ae634b6289ca55bb2b1016af => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:29.400300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca835d-e79d-4969-bbca-2725e39d6dcd None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:29.400675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca835d-e79d-4969-bbca-2725e39d6dcd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:29.400852 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca835d-e79d-4969-bbca-2725e39d6dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:29.425318 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2eca835d-e79d-4969-bbca-2725e39d6dcd None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:29.426955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2eca835d-e79d-4969-bbca-2725e39d6dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:29.482260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:29.491309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1106/2168] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:29.502025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:29.503784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:29.562304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a2d48-3d3a-41e4-8968-6da1eeff8038 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:29.565842 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069936ce-e4ea-491f-b264-d50e14817f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:29.566473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069936ce-e4ea-491f-b264-d50e14817f06 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:29.566754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069936ce-e4ea-491f-b264-d50e14817f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:29.567027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069936ce-e4ea-491f-b264-d50e14817f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:29.576034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069936ce-e4ea-491f-b264-d50e14817f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:29.577507 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069936ce-e4ea-491f-b264-d50e14817f06 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:29.588138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1063/2169] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:29.663968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757324a-44b2-42b5-a63d-a4c5d0ee2d96 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:29.664276 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757324a-44b2-42b5-a63d-a4c5d0ee2d96 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:29.664561 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757324a-44b2-42b5-a63d-a4c5d0ee2d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:29.686957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1757324a-44b2-42b5-a63d-a4c5d0ee2d96 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:29.688262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1757324a-44b2-42b5-a63d-a4c5d0ee2d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:29.745262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1107/2170] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:44:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:29.757692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069936ce-e4ea-491f-b264-d50e14817f06 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:29.759079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-069936ce-e4ea-491f-b264-d50e14817f06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:29.844233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069936ce-e4ea-491f-b264-d50e14817f06 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:29.850223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ea099-d43c-457e-86b2-7b6a12533791 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:29.850829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ea099-d43c-457e-86b2-7b6a12533791 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:29.851129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ea099-d43c-457e-86b2-7b6a12533791 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:29.851392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ea099-d43c-457e-86b2-7b6a12533791 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:29.859766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ea099-d43c-457e-86b2-7b6a12533791 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:29.861318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ea099-d43c-457e-86b2-7b6a12533791 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:29.872711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1064/2171] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:44:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:44:30.047267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757324a-44b2-42b5-a63d-a4c5d0ee2d96 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:30.050812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1757324a-44b2-42b5-a63d-a4c5d0ee2d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:30.065239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757324a-44b2-42b5-a63d-a4c5d0ee2d96 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:30.069887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:30.070576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:30.070858 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:30.071189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] PATH_INFO: `/v3/projects/2feaea4adb794093805a08f3a81cb33b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:30.072317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2feaea4adb794093805a08f3a81cb33b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:30.074263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:30.194875 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2feaea4adb794093805a08f3a81cb33b', 'request_id': 'req-a6259232-4c55-4766-a85f-0740d0b967f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:30.201113 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2feaea4adb794093805a08f3a81cb33b', 'request_id': 'req-a6259232-4c55-4766-a85f-0740d0b967f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:30.291865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2feaea4adb794093805a08f3a81cb33b', 'request_id': 'req-a6259232-4c55-4766-a85f-0740d0b967f8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:30.304264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] The token cache is being invalidate because project 2feaea4adb794093805a08f3a81cb33b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:30.304559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:30.305320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a6259232-4c55-4766-a85f-0740d0b967f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:30.307051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1108/2172] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:30 2022] DELETE /identity/v3/projects/2feaea4adb794093805a08f3a81cb33b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:30.310929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ea099-d43c-457e-86b2-7b6a12533791 service swift] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:30.322492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ea099-d43c-457e-86b2-7b6a12533791 service swift] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:30.379107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:30.379674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:30.379953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:30.380222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] PATH_INFO: `/v3/users/f4e22eb9e73e4a169ae63a50978d2beb` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:30.381326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e22eb9e73e4a169ae63a50978d2beb)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:30.383252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:30.619658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e22eb9e73e4a169ae63a50978d2beb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:30.620542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e22eb9e73e4a169ae63a50978d2beb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:30.641028 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e22eb9e73e4a169ae63a50978d2beb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:30.676773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:30.697397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e22eb9e73e4a169ae63a50978d2beb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:30.699418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e22eb9e73e4a169ae63a50978d2beb'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:30.723714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1065/2173] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:30 2022] DELETE /identity/v3/users/f4e22eb9e73e4a169ae63a50978d2beb => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:30.745998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff9378-065e-4e19-88b0-6b98ada2255e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:30.746386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff9378-065e-4e19-88b0-6b98ada2255e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:30.747004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff9378-065e-4e19-88b0-6b98ada2255e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:30.772273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9fff9378-065e-4e19-88b0-6b98ada2255e None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:30.773507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9fff9378-065e-4e19-88b0-6b98ada2255e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:30.838922 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1109/2174] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 05:44:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:30.857160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:30.857160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:30.928480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edf193-8a80-4e0d-add0-dbd4ea241f08 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:30.933828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6962039-9fb0-438c-9223-e16427e11aa6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:30.934534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6962039-9fb0-438c-9223-e16427e11aa6 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:30.934808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6962039-9fb0-438c-9223-e16427e11aa6 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:30.935075 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6962039-9fb0-438c-9223-e16427e11aa6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:30.944185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6962039-9fb0-438c-9223-e16427e11aa6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:30.945750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6962039-9fb0-438c-9223-e16427e11aa6 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:30.954024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1066/2175] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:44:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:44:31.217750 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff9378-065e-4e19-88b0-6b98ada2255e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:31.221406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9fff9378-065e-4e19-88b0-6b98ada2255e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:31.259204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:31.290934 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff9378-065e-4e19-88b0-6b98ada2255e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:31.295501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:31.296130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:31.296409 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:31.296683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] PATH_INFO: `/v3/projects/1542e6357c024d14ba1b5e830152ba67` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:31.297810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1542e6357c024d14ba1b5e830152ba67)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:31.299749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:31.417660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1542e6357c024d14ba1b5e830152ba67', 'request_id': 'req-53e70991-b16c-4c32-995f-f9824965c82c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:31.423858 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1542e6357c024d14ba1b5e830152ba67', 'request_id': 'req-53e70991-b16c-4c32-995f-f9824965c82c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:31.508412 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1542e6357c024d14ba1b5e830152ba67', 'request_id': 'req-53e70991-b16c-4c32-995f-f9824965c82c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:31.520701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] The token cache is being invalidate because project 1542e6357c024d14ba1b5e830152ba67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:31.520994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:31.521741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-53e70991-b16c-4c32-995f-f9824965c82c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:31.523671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1110/2176] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:31 2022] DELETE /identity/v3/projects/1542e6357c024d14ba1b5e830152ba67 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:31.529097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86cc7a-4902-4eb2-b1af-9b331729daac None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:31.529399 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86cc7a-4902-4eb2-b1af-9b331729daac None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:31.529691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86cc7a-4902-4eb2-b1af-9b331729daac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:31.549904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-bf86cc7a-4902-4eb2-b1af-9b331729daac None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:31.551664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bf86cc7a-4902-4eb2-b1af-9b331729daac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:31.619633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1067/2177] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:31.628038 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80cae4-90f2-42c7-92c6-6deb3f18ffc0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:31.628343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80cae4-90f2-42c7-92c6-6deb3f18ffc0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:31.628621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80cae4-90f2-42c7-92c6-6deb3f18ffc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:31.651219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7f80cae4-90f2-42c7-92c6-6deb3f18ffc0 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:31.652436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7f80cae4-90f2-42c7-92c6-6deb3f18ffc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:31.663255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1111/2178] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:31.667582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86cc7a-4902-4eb2-b1af-9b331729daac None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:31.678033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86cc7a-4902-4eb2-b1af-9b331729daac None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:31.682723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e7a77-516f-47f3-a271-9872feeb5ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:31.683450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e7a77-516f-47f3-a271-9872feeb5ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:31.683826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e7a77-516f-47f3-a271-9872feeb5ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:31.684218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e7a77-516f-47f3-a271-9872feeb5ba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:31.687222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e7a77-516f-47f3-a271-9872feeb5ba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:31.687222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e7a77-516f-47f3-a271-9872feeb5ba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:31.687222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e7a77-516f-47f3-a271-9872feeb5ba7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:31.703519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1068/2179] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:44:31.704604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80cae4-90f2-42c7-92c6-6deb3f18ffc0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:31.714385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80cae4-90f2-42c7-92c6-6deb3f18ffc0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:31.721713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e284a-fd99-470c-b148-7b1653c6654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:31.722401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e284a-fd99-470c-b148-7b1653c6654b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:31.722785 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e284a-fd99-470c-b148-7b1653c6654b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:31.723062 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e284a-fd99-470c-b148-7b1653c6654b admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:31.723753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e284a-fd99-470c-b148-7b1653c6654b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:31.725723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e284a-fd99-470c-b148-7b1653c6654b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:31.863381 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1112/2180] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:31 2022] POST /identity/v3/projects => generated 389 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:31.864719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e7a77-516f-47f3-a271-9872feeb5ba7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:31.878310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e7a77-516f-47f3-a271-9872feeb5ba7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:31.951758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782a326-5204-4e73-aacc-f00c67721cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:31.952399 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782a326-5204-4e73-aacc-f00c67721cbc admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:31.952685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782a326-5204-4e73-aacc-f00c67721cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:31.952959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782a326-5204-4e73-aacc-f00c67721cbc admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:31.953954 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782a326-5204-4e73-aacc-f00c67721cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:31.956030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782a326-5204-4e73-aacc-f00c67721cbc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:32.087006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1069/2181] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:31 2022] POST /identity/v3/users => generated 295 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:32.091432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e284a-fd99-470c-b148-7b1653c6654b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:32.102757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e284a-fd99-470c-b148-7b1653c6654b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:32.115362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4b92c-2309-4d53-bf34-bcf8f7c5a69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:32.115609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4b92c-2309-4d53-bf34-bcf8f7c5a69b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:32.116128 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4b92c-2309-4d53-bf34-bcf8f7c5a69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:32.116477 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4b92c-2309-4d53-bf34-bcf8f7c5a69b admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:32.117309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4b92c-2309-4d53-bf34-bcf8f7c5a69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:32.119293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4b92c-2309-4d53-bf34-bcf8f7c5a69b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:32.131371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1113/2182] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:32.135049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782a326-5204-4e73-aacc-f00c67721cbc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:32.146780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782a326-5204-4e73-aacc-f00c67721cbc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:32.152308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ac8a6-254d-4376-a74a-96694d245aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:32.153569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ac8a6-254d-4376-a74a-96694d245aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:32.154246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ac8a6-254d-4376-a74a-96694d245aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:32.154696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ac8a6-254d-4376-a74a-96694d245aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:32.155741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ac8a6-254d-4376-a74a-96694d245aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:32.157604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ac8a6-254d-4376-a74a-96694d245aaa admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:32.173675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1070/2183] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:44:32.176065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4b92c-2309-4d53-bf34-bcf8f7c5a69b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:32.187315 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4b92c-2309-4d53-bf34-bcf8f7c5a69b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:32.199535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f42c3c-8925-45bb-a77f-19584712898e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:32.200434 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f42c3c-8925-45bb-a77f-19584712898e admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:32.200803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f42c3c-8925-45bb-a77f-19584712898e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:32.201146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f42c3c-8925-45bb-a77f-19584712898e admin admin] PATH_INFO: `/v3/projects/58f953eb6e414342a433615e3e9802ee/users/a0c87c0a49bc4988b565fdc812ddf326/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:32.238613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f42c3c-8925-45bb-a77f-19584712898e admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f953eb6e414342a433615e3e9802ee, user_id=a0c87c0a49bc4988b565fdc812ddf326, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:32.241717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f42c3c-8925-45bb-a77f-19584712898e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:32.377683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1114/2184] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:32 2022] PUT /identity/v3/projects/58f953eb6e414342a433615e3e9802ee/users/a0c87c0a49bc4988b565fdc812ddf326/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:32.380440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe206b44-c328-405c-bb6d-58e3031b380d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:32.380789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe206b44-c328-405c-bb6d-58e3031b380d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:32.381188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe206b44-c328-405c-bb6d-58e3031b380d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:32.418624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fe206b44-c328-405c-bb6d-58e3031b380d None None] MFA Rules not processed for user `a0c87c0a49bc4988b565fdc812ddf326`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:32.420074 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fe206b44-c328-405c-bb6d-58e3031b380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:32.509660 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1071/2185] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:32.515523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f42c3c-8925-45bb-a77f-19584712898e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:32.602256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f42c3c-8925-45bb-a77f-19584712898e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:32.608998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948c3ed-62b6-427f-84b6-531d9f36fef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:32.609865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948c3ed-62b6-427f-84b6-531d9f36fef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:32.610390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948c3ed-62b6-427f-84b6-531d9f36fef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:32.610878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948c3ed-62b6-427f-84b6-531d9f36fef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:32.611531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948c3ed-62b6-427f-84b6-531d9f36fef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:32.613472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948c3ed-62b6-427f-84b6-531d9f36fef0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:32.753269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1115/2186] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:32 2022] POST /identity/v3/projects => generated 389 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:44:32.755043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe206b44-c328-405c-bb6d-58e3031b380d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:32.778822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe206b44-c328-405c-bb6d-58e3031b380d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:32.841625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb4be6-1c98-4227-89da-4f5b70aa50d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:32.842424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb4be6-1c98-4227-89da-4f5b70aa50d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:32.842764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb4be6-1c98-4227-89da-4f5b70aa50d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:32.843096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb4be6-1c98-4227-89da-4f5b70aa50d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:32.843816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb4be6-1c98-4227-89da-4f5b70aa50d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:32.845770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb4be6-1c98-4227-89da-4f5b70aa50d6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:32.951092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1072/2187] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:32 2022] POST /identity/v3/users => generated 301 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:32.955426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948c3ed-62b6-427f-84b6-531d9f36fef0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:32.966292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948c3ed-62b6-427f-84b6-531d9f36fef0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:32.970784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26b095-0778-44e0-8424-cc58d772c020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:32.971423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26b095-0778-44e0-8424-cc58d772c020 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:32.971720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26b095-0778-44e0-8424-cc58d772c020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:32.972024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26b095-0778-44e0-8424-cc58d772c020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:32.972670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26b095-0778-44e0-8424-cc58d772c020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:32.974425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26b095-0778-44e0-8424-cc58d772c020 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:32.985393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1116/2188] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:32.989612 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb4be6-1c98-4227-89da-4f5b70aa50d6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:32.999981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb4be6-1c98-4227-89da-4f5b70aa50d6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:33.005422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a03e-229d-4d15-bb71-fd2cb076b06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:33.006078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a03e-229d-4d15-bb71-fd2cb076b06b admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:33.006747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a03e-229d-4d15-bb71-fd2cb076b06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:33.007026 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a03e-229d-4d15-bb71-fd2cb076b06b admin admin] PATH_INFO: `/v3/projects/099197c67b574e7ab3196943c0b7dfa8/users/a039f56f356c4c9d8f84e260dec2f51a/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:33.024082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a03e-229d-4d15-bb71-fd2cb076b06b admin admin] RBAC: Authorizing `identity:create_grant(project_id=099197c67b574e7ab3196943c0b7dfa8, user_id=a039f56f356c4c9d8f84e260dec2f51a, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:33.026886 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a03e-229d-4d15-bb71-fd2cb076b06b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:33.147576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26b095-0778-44e0-8424-cc58d772c020 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:33.147780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1073/2189] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:32 2022] PUT /identity/v3/projects/099197c67b574e7ab3196943c0b7dfa8/users/a039f56f356c4c9d8f84e260dec2f51a/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:33.158952 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26b095-0778-44e0-8424-cc58d772c020 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:33.258339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e410f4f-6626-4139-ab6b-5f9a24fc20f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:33.259451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410f4f-6626-4139-ab6b-5f9a24fc20f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:33.262695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410f4f-6626-4139-ab6b-5f9a24fc20f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:33.262695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410f4f-6626-4139-ab6b-5f9a24fc20f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:33.262695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410f4f-6626-4139-ab6b-5f9a24fc20f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:33.262695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410f4f-6626-4139-ab6b-5f9a24fc20f2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:33.295851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1117/2190] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:33.300834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a03e-229d-4d15-bb71-fd2cb076b06b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:33.317838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a03e-229d-4d15-bb71-fd2cb076b06b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:33.339441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0a3e3-6feb-4982-8db7-8172d3577ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:33.341125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0a3e3-6feb-4982-8db7-8172d3577ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:33.341596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0a3e3-6feb-4982-8db7-8172d3577ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:33.342275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0a3e3-6feb-4982-8db7-8172d3577ccf admin admin] PATH_INFO: `/v3/projects/099197c67b574e7ab3196943c0b7dfa8/users/a039f56f356c4c9d8f84e260dec2f51a/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:33.344551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0a3e3-6feb-4982-8db7-8172d3577ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=099197c67b574e7ab3196943c0b7dfa8, user_id=a039f56f356c4c9d8f84e260dec2f51a, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:33.348807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0a3e3-6feb-4982-8db7-8172d3577ccf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:33.372567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1074/2191] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:33 2022] PUT /identity/v3/projects/099197c67b574e7ab3196943c0b7dfa8/users/a039f56f356c4c9d8f84e260dec2f51a/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:33.374972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e5033-603f-4ce3-9b74-fd838b5af15c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:33.375379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e5033-603f-4ce3-9b74-fd838b5af15c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:33.375717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e5033-603f-4ce3-9b74-fd838b5af15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:33.417442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-fc6e5033-603f-4ce3-9b74-fd838b5af15c None None] MFA Rules not processed for user `a039f56f356c4c9d8f84e260dec2f51a`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:33.418891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fc6e5033-603f-4ce3-9b74-fd838b5af15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:33.582670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1118/2192] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:33 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:33.594166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0a3e3-6feb-4982-8db7-8172d3577ccf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:33.596684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-61e0a3e3-6feb-4982-8db7-8172d3577ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:33.670671 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0a3e3-6feb-4982-8db7-8172d3577ccf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:33.683094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd344f87-b896-4baf-826d-d8fb52138d23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:33.683722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd344f87-b896-4baf-826d-d8fb52138d23 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:33.684003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd344f87-b896-4baf-826d-d8fb52138d23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:33.684299 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd344f87-b896-4baf-826d-d8fb52138d23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:33.695544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd344f87-b896-4baf-826d-d8fb52138d23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:33.697232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd344f87-b896-4baf-826d-d8fb52138d23 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:33.708575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1075/2193] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:33 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:33.723226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:33.885863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e5033-603f-4ce3-9b74-fd838b5af15c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:33.947479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd344f87-b896-4baf-826d-d8fb52138d23 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:33.947634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bd344f87-b896-4baf-826d-d8fb52138d23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:33.957064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e5033-603f-4ce3-9b74-fd838b5af15c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:33.961713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:33.961996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd344f87-b896-4baf-826d-d8fb52138d23 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:33.962758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:33.962995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:33.963267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] PATH_INFO: `/v3/users/a0c87c0a49bc4988b565fdc812ddf326` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:33.965582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0c87c0a49bc4988b565fdc812ddf326)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:33.967732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:33.969445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:33.970339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:33.970614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:33.970881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] PATH_INFO: `/v3/users/cadc251db5cf4e24a3fc8ffab6e9b9ae` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:33.972573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cadc251db5cf4e24a3fc8ffab6e9b9ae)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:33.974493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:34.197123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cadc251db5cf4e24a3fc8ffab6e9b9ae'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:34.197779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cadc251db5cf4e24a3fc8ffab6e9b9ae'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:34.214635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cadc251db5cf4e24a3fc8ffab6e9b9ae'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:34.217244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0c87c0a49bc4988b565fdc812ddf326'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:34.232166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0c87c0a49bc4988b565fdc812ddf326'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:34.233866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0c87c0a49bc4988b565fdc812ddf326'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:34.334435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cadc251db5cf4e24a3fc8ffab6e9b9ae'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:34.336979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e48c272c-036d-4673-97f6-71fa8ecee6d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cadc251db5cf4e24a3fc8ffab6e9b9ae'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:34.352087 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1076/2194] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:33 2022] DELETE /identity/v3/users/cadc251db5cf4e24a3fc8ffab6e9b9ae => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:34.356379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eced7-fb3e-4146-9f20-33b224bcb7dc None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:34.356646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eced7-fb3e-4146-9f20-33b224bcb7dc None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:34.356965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eced7-fb3e-4146-9f20-33b224bcb7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:34.375996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b12eced7-fb3e-4146-9f20-33b224bcb7dc None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:34.377204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b12eced7-fb3e-4146-9f20-33b224bcb7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:34.434344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0c87c0a49bc4988b565fdc812ddf326'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:34.435264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ad68b7c2-9f37-48fe-8952-cf4b486b098a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0c87c0a49bc4988b565fdc812ddf326'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:34.457714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1119/2195] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:33 2022] DELETE /identity/v3/users/a0c87c0a49bc4988b565fdc812ddf326 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:34.464338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee814dc-ed76-4630-997a-615d4fc67871 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:34.466116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee814dc-ed76-4630-997a-615d4fc67871 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:34.466529 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee814dc-ed76-4630-997a-615d4fc67871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:34.478704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1077/2196] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:34.491320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12eced7-fb3e-4146-9f20-33b224bcb7dc None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:34.492946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b12eced7-fb3e-4146-9f20-33b224bcb7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:34.493564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-cee814dc-ed76-4630-997a-615d4fc67871 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:34.495033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cee814dc-ed76-4630-997a-615d4fc67871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:34.508748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:34.552382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12eced7-fb3e-4146-9f20-33b224bcb7dc None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:34.568158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1120/2197] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:34.579736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee814dc-ed76-4630-997a-615d4fc67871 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:34.581671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-cee814dc-ed76-4630-997a-615d4fc67871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:34.644996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7e75e-61d7-419b-9e27-d29b2770f1f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:34.645867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7e75e-61d7-419b-9e27-d29b2770f1f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:34.646163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7e75e-61d7-419b-9e27-d29b2770f1f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:34.646433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7e75e-61d7-419b-9e27-d29b2770f1f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:34.658480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7e75e-61d7-419b-9e27-d29b2770f1f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:34.659947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7e75e-61d7-419b-9e27-d29b2770f1f3 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:34.670589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1078/2198] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:44:34.687084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee814dc-ed76-4630-997a-615d4fc67871 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:34.690712 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eea68-02a1-4b02-be45-3178723c10dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:34.691341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eea68-02a1-4b02-be45-3178723c10dc service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:34.691617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eea68-02a1-4b02-be45-3178723c10dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:34.691889 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eea68-02a1-4b02-be45-3178723c10dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:34.701613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eea68-02a1-4b02-be45-3178723c10dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:34.703065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eea68-02a1-4b02-be45-3178723c10dc service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:34.713064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1121/2199] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:35.032429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7e75e-61d7-419b-9e27-d29b2770f1f3 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:35.035449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-34b7e75e-61d7-419b-9e27-d29b2770f1f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:35.051347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7e75e-61d7-419b-9e27-d29b2770f1f3 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:35.056411 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:35.057051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.057326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.057916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] PATH_INFO: `/v3/projects/318c2ab91f564d8ba2e29b724c008dbc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.059031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=318c2ab91f564d8ba2e29b724c008dbc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:35.061220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:35.207559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eea68-02a1-4b02-be45-3178723c10dc service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:35.207559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-902eea68-02a1-4b02-be45-3178723c10dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:35.225536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318c2ab91f564d8ba2e29b724c008dbc', 'request_id': 'req-e0f84266-0f6e-4b87-ab81-902170458381'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.227985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eea68-02a1-4b02-be45-3178723c10dc service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:35.235331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318c2ab91f564d8ba2e29b724c008dbc', 'request_id': 'req-e0f84266-0f6e-4b87-ab81-902170458381'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.297843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:35.298514 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.298790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.299059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] PATH_INFO: `/v3/projects/58f953eb6e414342a433615e3e9802ee` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.300171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f953eb6e414342a433615e3e9802ee)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:35.302755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:35.330934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318c2ab91f564d8ba2e29b724c008dbc', 'request_id': 'req-e0f84266-0f6e-4b87-ab81-902170458381'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.342435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] The token cache is being invalidate because project 318c2ab91f564d8ba2e29b724c008dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:35.342724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.343368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e0f84266-0f6e-4b87-ab81-902170458381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.344919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1079/2200] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:35 2022] DELETE /identity/v3/projects/318c2ab91f564d8ba2e29b724c008dbc => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:35.356554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ceb64-be87-4adc-ac90-e0e214a3f5a8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.357040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ceb64-be87-4adc-ac90-e0e214a3f5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.357322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ceb64-be87-4adc-ac90-e0e214a3f5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.377622 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-440ceb64-be87-4adc-ac90-e0e214a3f5a8 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:35.378846 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-440ceb64-be87-4adc-ac90-e0e214a3f5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:35.386911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1080/2201] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:35.393934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595c622-c299-47f5-b6fc-1ce9593bb487 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.394267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595c622-c299-47f5-b6fc-1ce9593bb487 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.394539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595c622-c299-47f5-b6fc-1ce9593bb487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.415542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0595c622-c299-47f5-b6fc-1ce9593bb487 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:35.416701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0595c622-c299-47f5-b6fc-1ce9593bb487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:35.420069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f953eb6e414342a433615e3e9802ee', 'request_id': 'req-1e4b9be9-9904-4e41-94a4-c45a3363e91a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.427010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f953eb6e414342a433615e3e9802ee', 'request_id': 'req-1e4b9be9-9904-4e41-94a4-c45a3363e91a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.475254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1081/2202] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:35.479329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595c622-c299-47f5-b6fc-1ce9593bb487 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:35.488560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595c622-c299-47f5-b6fc-1ce9593bb487 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:35.494032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df185017-bb4a-4d52-aa20-7ef0d55dee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:35.495069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df185017-bb4a-4d52-aa20-7ef0d55dee77 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.502481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df185017-bb4a-4d52-aa20-7ef0d55dee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.502481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df185017-bb4a-4d52-aa20-7ef0d55dee77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.502481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df185017-bb4a-4d52-aa20-7ef0d55dee77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:35.502481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df185017-bb4a-4d52-aa20-7ef0d55dee77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:35.502481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df185017-bb4a-4d52-aa20-7ef0d55dee77 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:35.510768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1082/2203] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:44:35.514974 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df185017-bb4a-4d52-aa20-7ef0d55dee77 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:35.525136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df185017-bb4a-4d52-aa20-7ef0d55dee77 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:35.531522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f953eb6e414342a433615e3e9802ee', 'request_id': 'req-1e4b9be9-9904-4e41-94a4-c45a3363e91a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.534722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03965c1-4f22-442e-bf3e-fc29f1efd2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:35.535439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03965c1-4f22-442e-bf3e-fc29f1efd2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.536430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03965c1-4f22-442e-bf3e-fc29f1efd2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.536955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03965c1-4f22-442e-bf3e-fc29f1efd2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.537702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03965c1-4f22-442e-bf3e-fc29f1efd2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:35.539785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03965c1-4f22-442e-bf3e-fc29f1efd2b8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:35.544234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] The token cache is being invalidate because project 58f953eb6e414342a433615e3e9802ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:35.544527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.545178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.546708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1122/2204] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:35 2022] DELETE /identity/v3/projects/58f953eb6e414342a433615e3e9802ee => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:35.550641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:35.552114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:35.564891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b9be9-9904-4e41-94a4-c45a3363e91a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:35.569747 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:35.571564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.571564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.571564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] PATH_INFO: `/v3/users/a039f56f356c4c9d8f84e260dec2f51a` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.572411 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a039f56f356c4c9d8f84e260dec2f51a)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:35.574522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:35.626000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1083/2205] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:35 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:35.629964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03965c1-4f22-442e-bf3e-fc29f1efd2b8 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:35.631548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d03965c1-4f22-442e-bf3e-fc29f1efd2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:35.706580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03965c1-4f22-442e-bf3e-fc29f1efd2b8 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:35.712524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cf70a-6f20-4279-91d0-c9df90e7fa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:35.713138 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cf70a-6f20-4279-91d0-c9df90e7fa84 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.713408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cf70a-6f20-4279-91d0-c9df90e7fa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.713672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cf70a-6f20-4279-91d0-c9df90e7fa84 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.714318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cf70a-6f20-4279-91d0-c9df90e7fa84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:35.716310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cf70a-6f20-4279-91d0-c9df90e7fa84 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:35.812633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1084/2206] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:35 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:44:35.816703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cf70a-6f20-4279-91d0-c9df90e7fa84 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:35.819744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a039f56f356c4c9d8f84e260dec2f51a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.831706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cf70a-6f20-4279-91d0-c9df90e7fa84 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:35.836435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a039f56f356c4c9d8f84e260dec2f51a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.837676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a039f56f356c4c9d8f84e260dec2f51a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.838579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb80fd-ab62-435c-adf9-1d60d0f71ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:35.839193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb80fd-ab62-435c-adf9-1d60d0f71ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.839469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb80fd-ab62-435c-adf9-1d60d0f71ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.839738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb80fd-ab62-435c-adf9-1d60d0f71ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.840375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb80fd-ab62-435c-adf9-1d60d0f71ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:35.841989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb80fd-ab62-435c-adf9-1d60d0f71ce2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:35.852894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1085/2207] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:35.857213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb80fd-ab62-435c-adf9-1d60d0f71ce2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:35.858957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c5bb80fd-ab62-435c-adf9-1d60d0f71ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:35.871255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb80fd-ab62-435c-adf9-1d60d0f71ce2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:35.875895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95daa15-400e-4e6e-b5b0-247b41f43514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:35.876547 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95daa15-400e-4e6e-b5b0-247b41f43514 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.876889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95daa15-400e-4e6e-b5b0-247b41f43514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.877225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95daa15-400e-4e6e-b5b0-247b41f43514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.877924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95daa15-400e-4e6e-b5b0-247b41f43514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:35.880133 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95daa15-400e-4e6e-b5b0-247b41f43514 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:35.899482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1086/2208] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:44:35.901307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95daa15-400e-4e6e-b5b0-247b41f43514 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:35.911359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95daa15-400e-4e6e-b5b0-247b41f43514 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:35.916026 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b30e3-df10-4876-b1c6-0577eeac409f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:35.916678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b30e3-df10-4876-b1c6-0577eeac409f admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:35.916955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b30e3-df10-4876-b1c6-0577eeac409f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:35.917228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b30e3-df10-4876-b1c6-0577eeac409f admin admin] PATH_INFO: `/v3/projects/47a4c4de23fc4c3894545612c4386ad1/users/93201f697f2448d984a52b8c3958731a/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:35.931467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b30e3-df10-4876-b1c6-0577eeac409f admin admin] RBAC: Authorizing `identity:create_grant(project_id=47a4c4de23fc4c3894545612c4386ad1, user_id=93201f697f2448d984a52b8c3958731a, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:35.933527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a039f56f356c4c9d8f84e260dec2f51a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.937021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ca90b8e1-ac26-4b40-971b-073a21b0efbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a039f56f356c4c9d8f84e260dec2f51a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:35.940421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b30e3-df10-4876-b1c6-0577eeac409f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:35.950036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1123/2209] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:35 2022] DELETE /identity/v3/users/a039f56f356c4c9d8f84e260dec2f51a => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:36.019564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1087/2210] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:35 2022] PUT /identity/v3/projects/47a4c4de23fc4c3894545612c4386ad1/users/93201f697f2448d984a52b8c3958731a/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:36.030827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95772f9-0c11-4028-a083-4eea274d650e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:36.031330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95772f9-0c11-4028-a083-4eea274d650e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:36.031688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95772f9-0c11-4028-a083-4eea274d650e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:36.061178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d95772f9-0c11-4028-a083-4eea274d650e None None] MFA Rules not processed for user `93201f697f2448d984a52b8c3958731a`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:36.063019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d95772f9-0c11-4028-a083-4eea274d650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:36.130683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1124/2211] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:36.141559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b30e3-df10-4876-b1c6-0577eeac409f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:36.143450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4d7b30e3-df10-4876-b1c6-0577eeac409f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:36.151039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:36.208268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b30e3-df10-4876-b1c6-0577eeac409f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:36.212259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337106c6-dc3a-4215-8397-c9d631321447 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:36.212872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337106c6-dc3a-4215-8397-c9d631321447 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:36.213176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337106c6-dc3a-4215-8397-c9d631321447 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:36.213449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337106c6-dc3a-4215-8397-c9d631321447 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:36.221948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337106c6-dc3a-4215-8397-c9d631321447 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:36.227678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337106c6-dc3a-4215-8397-c9d631321447 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:36.233163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1088/2212] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:36.368421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:36.479280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95772f9-0c11-4028-a083-4eea274d650e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:36.480748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d95772f9-0c11-4028-a083-4eea274d650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:36.574839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95772f9-0c11-4028-a083-4eea274d650e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:36.581154 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:36.581809 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:36.582088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:36.582390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] PATH_INFO: `/v3/projects/099197c67b574e7ab3196943c0b7dfa8` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:36.588527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] RBAC: Authorizing `identity:delete_project(project_id=099197c67b574e7ab3196943c0b7dfa8)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:36.590671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:36.705936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '099197c67b574e7ab3196943c0b7dfa8', 'request_id': 'req-ac63d8ea-9530-4058-9c0f-9cb646c9919c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:36.713153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '099197c67b574e7ab3196943c0b7dfa8', 'request_id': 'req-ac63d8ea-9530-4058-9c0f-9cb646c9919c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:36.796653 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '099197c67b574e7ab3196943c0b7dfa8', 'request_id': 'req-ac63d8ea-9530-4058-9c0f-9cb646c9919c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:36.809394 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] The token cache is being invalidate because project 099197c67b574e7ab3196943c0b7dfa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:36.809958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:36.810818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ac63d8ea-9530-4058-9c0f-9cb646c9919c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:36.813273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1125/2213] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:36 2022] DELETE /identity/v3/projects/099197c67b574e7ab3196943c0b7dfa8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:36.823298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d0e3b-f13a-48be-b538-7155bed752f4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:36.823298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d0e3b-f13a-48be-b538-7155bed752f4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:36.823298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d0e3b-f13a-48be-b538-7155bed752f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:36.846162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-618d0e3b-f13a-48be-b538-7155bed752f4 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:36.847416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-618d0e3b-f13a-48be-b538-7155bed752f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:36.926131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1089/2214] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:36.933578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f334f4d-835a-4b43-be0e-5f35fa440ed4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:36.933860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f334f4d-835a-4b43-be0e-5f35fa440ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:36.934131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f334f4d-835a-4b43-be0e-5f35fa440ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:36.955221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2f334f4d-835a-4b43-be0e-5f35fa440ed4 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:36.959520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2f334f4d-835a-4b43-be0e-5f35fa440ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:36.970339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1126/2215] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:36.978349 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d0e3b-f13a-48be-b538-7155bed752f4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:36.988511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d0e3b-f13a-48be-b538-7155bed752f4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:36.993523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802160f-608c-4f61-a9bb-8d7696320578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:36.994128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802160f-608c-4f61-a9bb-8d7696320578 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:36.994441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802160f-608c-4f61-a9bb-8d7696320578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:36.994712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802160f-608c-4f61-a9bb-8d7696320578 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:36.995391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802160f-608c-4f61-a9bb-8d7696320578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:36.995695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802160f-608c-4f61-a9bb-8d7696320578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:36.997285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802160f-608c-4f61-a9bb-8d7696320578 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:37.014108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1090/2216] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:44:37.014819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f334f4d-835a-4b43-be0e-5f35fa440ed4 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:37.025714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f334f4d-835a-4b43-be0e-5f35fa440ed4 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:37.057174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4451741-d9ce-485c-9305-fd42491079d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:37.057892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4451741-d9ce-485c-9305-fd42491079d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:37.058267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4451741-d9ce-485c-9305-fd42491079d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:37.058610 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4451741-d9ce-485c-9305-fd42491079d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:37.059351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4451741-d9ce-485c-9305-fd42491079d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:37.061339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4451741-d9ce-485c-9305-fd42491079d7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:37.196470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1127/2217] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:37 2022] POST /identity/v3/projects => generated 381 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:44:37.197769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802160f-608c-4f61-a9bb-8d7696320578 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:37.208310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802160f-608c-4f61-a9bb-8d7696320578 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:37.248057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2471204-e4ea-4ae8-a6fd-1e6b820661ed None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:37.248347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2471204-e4ea-4ae8-a6fd-1e6b820661ed None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:37.248624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2471204-e4ea-4ae8-a6fd-1e6b820661ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:37.272604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e2471204-e4ea-4ae8-a6fd-1e6b820661ed None None] MFA Rules not processed for user `c66c7ddbf18d4f1fb8c890f548031f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:37.273877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e2471204-e4ea-4ae8-a6fd-1e6b820661ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:37.301137 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fef77-72ef-407d-b535-67724c1fb93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:37.302005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fef77-72ef-407d-b535-67724c1fb93b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:37.302518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fef77-72ef-407d-b535-67724c1fb93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:37.302801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fef77-72ef-407d-b535-67724c1fb93b admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:37.303514 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570fef77-72ef-407d-b535-67724c1fb93b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:37.305428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570fef77-72ef-407d-b535-67724c1fb93b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:37.335417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1128/2218] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:44:37.373455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:37.410976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1091/2219] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:37 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:37.415057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2471204-e4ea-4ae8-a6fd-1e6b820661ed None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:37.425480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2471204-e4ea-4ae8-a6fd-1e6b820661ed None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:37.430265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b15cf-cc55-403d-bce7-d162b2cbc7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:37.430867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b15cf-cc55-403d-bce7-d162b2cbc7da admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:37.434365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b15cf-cc55-403d-bce7-d162b2cbc7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:37.434365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b15cf-cc55-403d-bce7-d162b2cbc7da admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:37.434365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b15cf-cc55-403d-bce7-d162b2cbc7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:37.434365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b15cf-cc55-403d-bce7-d162b2cbc7da admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:37.444233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1129/2220] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:44:37.448078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fef77-72ef-407d-b535-67724c1fb93b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:37.458894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fef77-72ef-407d-b535-67724c1fb93b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:37.463685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99fd60-5d56-40a5-9592-4300acf0b2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:37.464308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99fd60-5d56-40a5-9592-4300acf0b2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:37.464590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99fd60-5d56-40a5-9592-4300acf0b2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:37.468505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99fd60-5d56-40a5-9592-4300acf0b2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:37.468505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99fd60-5d56-40a5-9592-4300acf0b2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:37.468505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99fd60-5d56-40a5-9592-4300acf0b2d9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:37.477733 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1092/2221] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:37.481964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b15cf-cc55-403d-bce7-d162b2cbc7da admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:37.491863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b15cf-cc55-403d-bce7-d162b2cbc7da admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:37.496443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88738e6e-2b2f-4cf3-9f7c-1e234013a47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:37.497097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88738e6e-2b2f-4cf3-9f7c-1e234013a47b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:37.497373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88738e6e-2b2f-4cf3-9f7c-1e234013a47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:37.497658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88738e6e-2b2f-4cf3-9f7c-1e234013a47b admin admin] PATH_INFO: `/v3/projects/8ab753fc8ea642f0b1e8a57966fa758e/users/b43bb2eab43a4fc18d988edf62180060/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:37.513923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88738e6e-2b2f-4cf3-9f7c-1e234013a47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab753fc8ea642f0b1e8a57966fa758e, user_id=b43bb2eab43a4fc18d988edf62180060, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:37.516830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88738e6e-2b2f-4cf3-9f7c-1e234013a47b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:37.660799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1130/2222] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:37 2022] PUT /identity/v3/projects/8ab753fc8ea642f0b1e8a57966fa758e/users/b43bb2eab43a4fc18d988edf62180060/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:37.675244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cab68d-1e80-42eb-963b-555299e3c72f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:37.675244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cab68d-1e80-42eb-963b-555299e3c72f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:37.675244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cab68d-1e80-42eb-963b-555299e3c72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:37.702083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-54cab68d-1e80-42eb-963b-555299e3c72f None None] MFA Rules not processed for user `b43bb2eab43a4fc18d988edf62180060`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:37.703319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-54cab68d-1e80-42eb-963b-555299e3c72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:37.773044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1093/2223] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:37.778560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88738e6e-2b2f-4cf3-9f7c-1e234013a47b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:37.861517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88738e6e-2b2f-4cf3-9f7c-1e234013a47b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:37.866748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc6eb0-da84-4bd0-a70f-3f67a39039c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:37.867442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc6eb0-da84-4bd0-a70f-3f67a39039c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:37.867711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc6eb0-da84-4bd0-a70f-3f67a39039c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:37.867982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc6eb0-da84-4bd0-a70f-3f67a39039c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:37.868620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc6eb0-da84-4bd0-a70f-3f67a39039c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:37.870649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc6eb0-da84-4bd0-a70f-3f67a39039c0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:38.001365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:38.013765 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1131/2224] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:37 2022] POST /identity/v3/projects => generated 381 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:38.017984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cab68d-1e80-42eb-963b-555299e3c72f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:38.029352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cab68d-1e80-42eb-963b-555299e3c72f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:38.137151 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9ae82-d4bc-4523-82f5-fa7f4955e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:38.138166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9ae82-d4bc-4523-82f5-fa7f4955e38c admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:38.138442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9ae82-d4bc-4523-82f5-fa7f4955e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:38.138767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9ae82-d4bc-4523-82f5-fa7f4955e38c admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:38.139814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9ae82-d4bc-4523-82f5-fa7f4955e38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:38.141914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9ae82-d4bc-4523-82f5-fa7f4955e38c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:38.230411 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1094/2225] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:38 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:38.232368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc6eb0-da84-4bd0-a70f-3f67a39039c0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:38.248140 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc6eb0-da84-4bd0-a70f-3f67a39039c0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:38.252837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453dab3-a1d9-4469-8b4e-2e67b5a73969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:38.254068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453dab3-a1d9-4469-8b4e-2e67b5a73969 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:38.254925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453dab3-a1d9-4469-8b4e-2e67b5a73969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:38.255209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453dab3-a1d9-4469-8b4e-2e67b5a73969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:38.255972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453dab3-a1d9-4469-8b4e-2e67b5a73969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:38.263700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453dab3-a1d9-4469-8b4e-2e67b5a73969 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:38.276867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1132/2226] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:38.278453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9ae82-d4bc-4523-82f5-fa7f4955e38c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:38.290448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9ae82-d4bc-4523-82f5-fa7f4955e38c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:38.295858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730b029-3723-44d4-b451-03e6ca1cd8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:38.296562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730b029-3723-44d4-b451-03e6ca1cd8f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:38.296836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730b029-3723-44d4-b451-03e6ca1cd8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:38.297131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730b029-3723-44d4-b451-03e6ca1cd8f5 admin admin] PATH_INFO: `/v3/projects/168ad2b807c14ced8f0f6f1628a49d53/users/6765701836e743b2af77815e5dfc9030/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:38.315174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730b029-3723-44d4-b451-03e6ca1cd8f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=168ad2b807c14ced8f0f6f1628a49d53, user_id=6765701836e743b2af77815e5dfc9030, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:38.320055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730b029-3723-44d4-b451-03e6ca1cd8f5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:38.417985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1095/2227] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:38 2022] PUT /identity/v3/projects/168ad2b807c14ced8f0f6f1628a49d53/users/6765701836e743b2af77815e5dfc9030/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:44:38.419257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453dab3-a1d9-4469-8b4e-2e67b5a73969 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:38.432071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453dab3-a1d9-4469-8b4e-2e67b5a73969 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:38.496362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8129035-fe1d-4414-88ec-4ce0963f471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:38.497037 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129035-fe1d-4414-88ec-4ce0963f471f admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:38.497283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129035-fe1d-4414-88ec-4ce0963f471f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:38.497556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8129035-fe1d-4414-88ec-4ce0963f471f admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:38.498270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8129035-fe1d-4414-88ec-4ce0963f471f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:38.499863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8129035-fe1d-4414-88ec-4ce0963f471f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:38.510628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1133/2228] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:38.514588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730b029-3723-44d4-b451-03e6ca1cd8f5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:38.524819 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730b029-3723-44d4-b451-03e6ca1cd8f5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:38.529340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a2306-37c2-49b3-b628-d05a58e6b092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:38.529972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a2306-37c2-49b3-b628-d05a58e6b092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:38.530279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a2306-37c2-49b3-b628-d05a58e6b092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:38.531790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a2306-37c2-49b3-b628-d05a58e6b092 admin admin] PATH_INFO: `/v3/projects/168ad2b807c14ced8f0f6f1628a49d53/users/6765701836e743b2af77815e5dfc9030/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:38.532614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a2306-37c2-49b3-b628-d05a58e6b092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=168ad2b807c14ced8f0f6f1628a49d53, user_id=6765701836e743b2af77815e5dfc9030, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:38.535299 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a2306-37c2-49b3-b628-d05a58e6b092 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:38.545798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1096/2229] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:38 2022] PUT /identity/v3/projects/168ad2b807c14ced8f0f6f1628a49d53/users/6765701836e743b2af77815e5dfc9030/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:38.556797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa4a46-42be-4ce6-b41e-e283b652ecff None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:38.559357 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa4a46-42be-4ce6-b41e-e283b652ecff None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:38.559357 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa4a46-42be-4ce6-b41e-e283b652ecff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:38.585353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-65aa4a46-42be-4ce6-b41e-e283b652ecff None None] MFA Rules not processed for user `6765701836e743b2af77815e5dfc9030`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:38.586606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-65aa4a46-42be-4ce6-b41e-e283b652ecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:38.654146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1134/2230] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:38.879609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a2306-37c2-49b3-b628-d05a58e6b092 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:38.881396 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-be5a2306-37c2-49b3-b628-d05a58e6b092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:38.945192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a2306-37c2-49b3-b628-d05a58e6b092 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:38.948909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89d27b-e357-480d-8f8e-8cb78f836655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:38.949506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89d27b-e357-480d-8f8e-8cb78f836655 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:38.949784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89d27b-e357-480d-8f8e-8cb78f836655 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:38.950056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89d27b-e357-480d-8f8e-8cb78f836655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:38.958869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89d27b-e357-480d-8f8e-8cb78f836655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:38.960281 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89d27b-e357-480d-8f8e-8cb78f836655 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:38.969410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1097/2231] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:39.170657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa4a46-42be-4ce6-b41e-e283b652ecff None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:39.180045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa4a46-42be-4ce6-b41e-e283b652ecff None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:39.183770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b1511-684b-4ef3-b5d7-0ab73998772d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:39.184400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b1511-684b-4ef3-b5d7-0ab73998772d service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:39.184981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b1511-684b-4ef3-b5d7-0ab73998772d service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:39.185257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b1511-684b-4ef3-b5d7-0ab73998772d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:39.243686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b1511-684b-4ef3-b5d7-0ab73998772d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:39.245484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b1511-684b-4ef3-b5d7-0ab73998772d service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:39.254503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1135/2232] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:39.510331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:39.630505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:39.798706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89d27b-e357-480d-8f8e-8cb78f836655 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:39.865134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89d27b-e357-480d-8f8e-8cb78f836655 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:39.870011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:39.870748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:39.871023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:39.871292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] PATH_INFO: `/v3/users/b43bb2eab43a4fc18d988edf62180060` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:39.872456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b43bb2eab43a4fc18d988edf62180060)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:39.874487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:40.049243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b43bb2eab43a4fc18d988edf62180060'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:40.049960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b43bb2eab43a4fc18d988edf62180060'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:40.063280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b43bb2eab43a4fc18d988edf62180060'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:40.121582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b43bb2eab43a4fc18d988edf62180060'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:40.122226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b43bb2eab43a4fc18d988edf62180060'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:40.135569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1098/2233] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:39 2022] DELETE /identity/v3/users/b43bb2eab43a4fc18d988edf62180060 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:40.140111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6fc1c-1f02-4326-8f72-35ab5c7bec6e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:40.140420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6fc1c-1f02-4326-8f72-35ab5c7bec6e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:40.140693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6fc1c-1f02-4326-8f72-35ab5c7bec6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:40.159106 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ecb6fc1c-1f02-4326-8f72-35ab5c7bec6e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:40.160302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ecb6fc1c-1f02-4326-8f72-35ab5c7bec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:40.216540 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1136/2234] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:40.226048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:40.227452 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:40.282849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90fc06-cfba-44a5-91f0-908b2e183f6b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:40.287038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e371052-849a-41ef-97ff-12d19bd93f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:40.287636 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e371052-849a-41ef-97ff-12d19bd93f3d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:40.287910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e371052-849a-41ef-97ff-12d19bd93f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:40.288177 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e371052-849a-41ef-97ff-12d19bd93f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:40.296848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e371052-849a-41ef-97ff-12d19bd93f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:40.298220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e371052-849a-41ef-97ff-12d19bd93f3d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:40.307900 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1099/2235] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:40.327773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:40.385019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:40.616184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:40.756731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:40.797327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6fc1c-1f02-4326-8f72-35ab5c7bec6e None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:40.799008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ecb6fc1c-1f02-4326-8f72-35ab5c7bec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:40.802491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:40.812145 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6fc1c-1f02-4326-8f72-35ab5c7bec6e None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:40.817101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:40.817813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:40.818216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:40.818656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] PATH_INFO: `/v3/projects/8ab753fc8ea642f0b1e8a57966fa758e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:40.819810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab753fc8ea642f0b1e8a57966fa758e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:40.821945 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:40.945123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab753fc8ea642f0b1e8a57966fa758e', 'request_id': 'req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:40.952425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab753fc8ea642f0b1e8a57966fa758e', 'request_id': 'req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:41.024269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab753fc8ea642f0b1e8a57966fa758e', 'request_id': 'req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:41.035138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] The token cache is being invalidate because project 8ab753fc8ea642f0b1e8a57966fa758e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:41.035443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:41.036050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:41.037595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1137/2236] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:40 2022] DELETE /identity/v3/projects/8ab753fc8ea642f0b1e8a57966fa758e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:41.041836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e371052-849a-41ef-97ff-12d19bd93f3d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:41.051352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e371052-849a-41ef-97ff-12d19bd93f3d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:41.079606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:41.108295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:41.108973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:41.109450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:41.109893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] PATH_INFO: `/v3/users/6765701836e743b2af77815e5dfc9030` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:41.111610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:41.112473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] RBAC: Authorizing `identity:delete_user(user_id=6765701836e743b2af77815e5dfc9030)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:41.114520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:41.318418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6765701836e743b2af77815e5dfc9030'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:41.319073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6765701836e743b2af77815e5dfc9030'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:41.331053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6765701836e743b2af77815e5dfc9030'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:41.406811 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6765701836e743b2af77815e5dfc9030'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:41.407421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dafb7db9-c047-47f1-a2dc-ed2275de10df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6765701836e743b2af77815e5dfc9030'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:41.419978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1100/2237] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:41 2022] DELETE /identity/v3/users/6765701836e743b2af77815e5dfc9030 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:41.894435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:41.896173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:41.915764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b64b37-9a7c-49f2-9e45-e2b5193718ab admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:42.008095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:42.008820 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:42.009106 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:42.009404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] PATH_INFO: `/v3/projects/168ad2b807c14ced8f0f6f1628a49d53` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:42.014070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=168ad2b807c14ced8f0f6f1628a49d53)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:42.014070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:42.142450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168ad2b807c14ced8f0f6f1628a49d53', 'request_id': 'req-2433e852-4335-4e06-96ec-11acec79bccb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:42.149211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168ad2b807c14ced8f0f6f1628a49d53', 'request_id': 'req-2433e852-4335-4e06-96ec-11acec79bccb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:42.212770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168ad2b807c14ced8f0f6f1628a49d53', 'request_id': 'req-2433e852-4335-4e06-96ec-11acec79bccb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:42.224545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] The token cache is being invalidate because project 168ad2b807c14ced8f0f6f1628a49d53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:42.224834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:42.225453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-2433e852-4335-4e06-96ec-11acec79bccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:42.226984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1138/2238] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:41 2022] DELETE /identity/v3/projects/168ad2b807c14ced8f0f6f1628a49d53 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:42.232000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21854909-b13d-4d6e-ac45-9e879537481f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:42.232473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21854909-b13d-4d6e-ac45-9e879537481f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:42.232571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21854909-b13d-4d6e-ac45-9e879537481f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:42.251659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-21854909-b13d-4d6e-ac45-9e879537481f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:42.252862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-21854909-b13d-4d6e-ac45-9e879537481f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:42.310499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1101/2239] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:42.319591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a1f00-8e85-4462-87d4-663c8197fbbc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:42.319877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a1f00-8e85-4462-87d4-663c8197fbbc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:42.320152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a1f00-8e85-4462-87d4-663c8197fbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:42.339977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-836a1f00-8e85-4462-87d4-663c8197fbbc None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:42.343805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-836a1f00-8e85-4462-87d4-663c8197fbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:42.349986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1139/2240] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:42.354298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21854909-b13d-4d6e-ac45-9e879537481f None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:42.364234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21854909-b13d-4d6e-ac45-9e879537481f None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:42.368996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c273177-1604-4d63-a57e-96ccd6650f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:42.369622 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c273177-1604-4d63-a57e-96ccd6650f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:42.369901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c273177-1604-4d63-a57e-96ccd6650f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:42.370194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c273177-1604-4d63-a57e-96ccd6650f63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:42.370845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c273177-1604-4d63-a57e-96ccd6650f63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:42.371149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c273177-1604-4d63-a57e-96ccd6650f63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:42.372731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c273177-1604-4d63-a57e-96ccd6650f63 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:42.384413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1102/2241] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:44:42.388402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a1f00-8e85-4462-87d4-663c8197fbbc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:42.397962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a1f00-8e85-4462-87d4-663c8197fbbc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:42.402511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5fe0c-65cd-4c7c-822f-f46cf8f29cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:42.403096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5fe0c-65cd-4c7c-822f-f46cf8f29cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:42.403372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5fe0c-65cd-4c7c-822f-f46cf8f29cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:42.403643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5fe0c-65cd-4c7c-822f-f46cf8f29cd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:42.404269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5fe0c-65cd-4c7c-822f-f46cf8f29cd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:42.406170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5fe0c-65cd-4c7c-822f-f46cf8f29cd0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:42.487622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:42.494646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1140/2242] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:42 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:44:42.498899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c273177-1604-4d63-a57e-96ccd6650f63 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:42.509370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c273177-1604-4d63-a57e-96ccd6650f63 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:42.553542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:42.565111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babff1c-4941-419b-9aa3-30e67b3081f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:42.565709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babff1c-4941-419b-9aa3-30e67b3081f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:42.565987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babff1c-4941-419b-9aa3-30e67b3081f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:42.566297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babff1c-4941-419b-9aa3-30e67b3081f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:42.567473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babff1c-4941-419b-9aa3-30e67b3081f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:42.569412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babff1c-4941-419b-9aa3-30e67b3081f6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:42.651967 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1103/2243] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:42 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:42.656216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5fe0c-65cd-4c7c-822f-f46cf8f29cd0 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:42.666998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5fe0c-65cd-4c7c-822f-f46cf8f29cd0 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:42.671919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b09d1-8189-4a3a-b025-b05526c83d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:42.672529 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b09d1-8189-4a3a-b025-b05526c83d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:42.672810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b09d1-8189-4a3a-b025-b05526c83d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:42.673087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b09d1-8189-4a3a-b025-b05526c83d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:42.673748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b09d1-8189-4a3a-b025-b05526c83d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:42.675532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b09d1-8189-4a3a-b025-b05526c83d87 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:42.686509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1141/2244] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:42.690687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babff1c-4941-419b-9aa3-30e67b3081f6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:42.701503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babff1c-4941-419b-9aa3-30e67b3081f6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:42.708332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd541b-2541-4c59-8c48-4a7c4ced6a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:42.708952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd541b-2541-4c59-8c48-4a7c4ced6a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:42.709233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd541b-2541-4c59-8c48-4a7c4ced6a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:42.709506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd541b-2541-4c59-8c48-4a7c4ced6a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:42.710160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd541b-2541-4c59-8c48-4a7c4ced6a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:42.711787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd541b-2541-4c59-8c48-4a7c4ced6a0c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:42.722385 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1104/2245] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:42.726859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b09d1-8189-4a3a-b025-b05526c83d87 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:42.738526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b09d1-8189-4a3a-b025-b05526c83d87 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:42.743445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ac35-44f8-4a4b-a114-5ccca346f802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:42.744007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ac35-44f8-4a4b-a114-5ccca346f802 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:42.744287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ac35-44f8-4a4b-a114-5ccca346f802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:42.744688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ac35-44f8-4a4b-a114-5ccca346f802 admin admin] PATH_INFO: `/v3/projects/7fc58f79e84447a5a2158c7328ac263e/users/3b847ccda37a48d593cd026ba5532932/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:42.763737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307ac35-44f8-4a4b-a114-5ccca346f802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc58f79e84447a5a2158c7328ac263e, user_id=3b847ccda37a48d593cd026ba5532932, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:42.766711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307ac35-44f8-4a4b-a114-5ccca346f802 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:42.911886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1142/2246] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:42 2022] PUT /identity/v3/projects/7fc58f79e84447a5a2158c7328ac263e/users/3b847ccda37a48d593cd026ba5532932/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:42.922287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02274d2d-9dd4-4b2e-a21f-48d25a9318ed None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:42.922579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02274d2d-9dd4-4b2e-a21f-48d25a9318ed None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:42.922859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02274d2d-9dd4-4b2e-a21f-48d25a9318ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:42.955945 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-02274d2d-9dd4-4b2e-a21f-48d25a9318ed None None] MFA Rules not processed for user `3b847ccda37a48d593cd026ba5532932`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:42.957197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-02274d2d-9dd4-4b2e-a21f-48d25a9318ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:43.042478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1105/2247] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:43.053924 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ac35-44f8-4a4b-a114-5ccca346f802 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:43.055641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9307ac35-44f8-4a4b-a114-5ccca346f802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:43.121908 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ac35-44f8-4a4b-a114-5ccca346f802 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:43.127990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49015097-29af-426b-8eed-b38c6392b24e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:43.129877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015097-29af-426b-8eed-b38c6392b24e service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:43.129877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015097-29af-426b-8eed-b38c6392b24e service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:43.130180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015097-29af-426b-8eed-b38c6392b24e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:43.141971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49015097-29af-426b-8eed-b38c6392b24e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:43.143620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49015097-29af-426b-8eed-b38c6392b24e service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:43.153855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1143/2248] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:53.059670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:53.149122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:56.236656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02274d2d-9dd4-4b2e-a21f-48d25a9318ed None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:56.336869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02274d2d-9dd4-4b2e-a21f-48d25a9318ed None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:56.343261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:56.344879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:56.345330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:56.345803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] PATH_INFO: `/v3/users/3b847ccda37a48d593cd026ba5532932` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:56.349050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b847ccda37a48d593cd026ba5532932)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:56.354222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:56.622203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b847ccda37a48d593cd026ba5532932'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:56.622203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b847ccda37a48d593cd026ba5532932'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:56.638525 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b847ccda37a48d593cd026ba5532932'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:56.690893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b847ccda37a48d593cd026ba5532932'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:56.692485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b847ccda37a48d593cd026ba5532932'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:56.707682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1106/2249] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:56 2022] DELETE /identity/v3/users/3b847ccda37a48d593cd026ba5532932 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:56.712390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca47557-3648-467f-87c8-450c62596b52 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:56.713222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca47557-3648-467f-87c8-450c62596b52 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:56.713496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca47557-3648-467f-87c8-450c62596b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:56.732928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-dca47557-3648-467f-87c8-450c62596b52 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:56.734293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dca47557-3648-467f-87c8-450c62596b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:56.797701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1144/2250] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:56.807903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:56.809546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:56.872229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d315056-b37d-40a5-882f-a5f9dd4965f6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:56.875871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852ba7b-3003-459c-a792-18ba2b107f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:56.876476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852ba7b-3003-459c-a792-18ba2b107f11 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:56.876762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852ba7b-3003-459c-a792-18ba2b107f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:56.877038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852ba7b-3003-459c-a792-18ba2b107f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:56.887914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852ba7b-3003-459c-a792-18ba2b107f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:56.889416 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852ba7b-3003-459c-a792-18ba2b107f11 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:56.900964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1107/2251] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:44:57.789396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca47557-3648-467f-87c8-450c62596b52 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:57.791403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-dca47557-3648-467f-87c8-450c62596b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:57.804941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca47557-3648-467f-87c8-450c62596b52 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:57.810574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:57.811759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:57.812250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:57.812527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] PATH_INFO: `/v3/projects/7fc58f79e84447a5a2158c7328ac263e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:57.813716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc58f79e84447a5a2158c7328ac263e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:57.816525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:57.973328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc58f79e84447a5a2158c7328ac263e', 'request_id': 'req-a68503c9-d732-485b-99fd-36a7131b52a9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:57.983637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc58f79e84447a5a2158c7328ac263e', 'request_id': 'req-a68503c9-d732-485b-99fd-36a7131b52a9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:58.080990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc58f79e84447a5a2158c7328ac263e', 'request_id': 'req-a68503c9-d732-485b-99fd-36a7131b52a9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:58.095210 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] The token cache is being invalidate because project 7fc58f79e84447a5a2158c7328ac263e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:44:58.095723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:58.096836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a68503c9-d732-485b-99fd-36a7131b52a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:44:58.103459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1145/2252] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:44:57 2022] DELETE /identity/v3/projects/7fc58f79e84447a5a2158c7328ac263e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:58.104794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b199-297b-4707-a707-0891c4f02e93 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:58.105291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b199-297b-4707-a707-0891c4f02e93 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:58.105388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b199-297b-4707-a707-0891c4f02e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:58.135779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ad21b199-297b-4707-a707-0891c4f02e93 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:58.137321 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ad21b199-297b-4707-a707-0891c4f02e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:58.175993 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:58.245347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1108/2253] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:58.255758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54bb5b-b2dc-46a2-9193-5ec05415fe6c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:58.256050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54bb5b-b2dc-46a2-9193-5ec05415fe6c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:58.256333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54bb5b-b2dc-46a2-9193-5ec05415fe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:58.288482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7c54bb5b-b2dc-46a2-9193-5ec05415fe6c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:58.289845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7c54bb5b-b2dc-46a2-9193-5ec05415fe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:58.300114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1146/2254] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:58.305105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b199-297b-4707-a707-0891c4f02e93 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:58.318510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b199-297b-4707-a707-0891c4f02e93 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:58.325904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdba687-217d-44d6-a7db-c7d94a012fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:58.326554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdba687-217d-44d6-a7db-c7d94a012fef admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:58.326838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdba687-217d-44d6-a7db-c7d94a012fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:58.327109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdba687-217d-44d6-a7db-c7d94a012fef admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:58.328365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdba687-217d-44d6-a7db-c7d94a012fef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:44:58.328722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdba687-217d-44d6-a7db-c7d94a012fef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:58.330374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdba687-217d-44d6-a7db-c7d94a012fef admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:58.349613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1109/2255] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:44:58.350823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54bb5b-b2dc-46a2-9193-5ec05415fe6c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:58.360904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54bb5b-b2dc-46a2-9193-5ec05415fe6c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:58.369536 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae25742-2217-4428-b16d-87b88eca2eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:58.370242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae25742-2217-4428-b16d-87b88eca2eac admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:58.370956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae25742-2217-4428-b16d-87b88eca2eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:58.370956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae25742-2217-4428-b16d-87b88eca2eac admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:58.371545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae25742-2217-4428-b16d-87b88eca2eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:58.373601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae25742-2217-4428-b16d-87b88eca2eac admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:58.463387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1147/2256] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:44:58 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:58.467579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdba687-217d-44d6-a7db-c7d94a012fef admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:58.478267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdba687-217d-44d6-a7db-c7d94a012fef admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:58.540887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d0e8e-c8a3-426a-ab12-a403e866a117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:58.541497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d0e8e-c8a3-426a-ab12-a403e866a117 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:58.541775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d0e8e-c8a3-426a-ab12-a403e866a117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:58.542045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d0e8e-c8a3-426a-ab12-a403e866a117 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:58.542918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d0e8e-c8a3-426a-ab12-a403e866a117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:58.545018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d0e8e-c8a3-426a-ab12-a403e866a117 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:58.678985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1110/2257] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:44:58 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:44:58.683238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae25742-2217-4428-b16d-87b88eca2eac admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:58.693808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae25742-2217-4428-b16d-87b88eca2eac admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:58.698309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc3b83-7062-4cde-a1c8-1a1d466624e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:58.698900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc3b83-7062-4cde-a1c8-1a1d466624e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:58.699173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc3b83-7062-4cde-a1c8-1a1d466624e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:58.699440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc3b83-7062-4cde-a1c8-1a1d466624e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:58.700076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc3b83-7062-4cde-a1c8-1a1d466624e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:58.701660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc3b83-7062-4cde-a1c8-1a1d466624e8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:58.711838 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1148/2258] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:58.715797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d0e8e-c8a3-426a-ab12-a403e866a117 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:58.726390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d0e8e-c8a3-426a-ab12-a403e866a117 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:58.731189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce89407-2618-4d16-a01c-2f7810a4b4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:58.731782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce89407-2618-4d16-a01c-2f7810a4b4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:58.732061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce89407-2618-4d16-a01c-2f7810a4b4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:58.732333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce89407-2618-4d16-a01c-2f7810a4b4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:58.732972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce89407-2618-4d16-a01c-2f7810a4b4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:58.734582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce89407-2618-4d16-a01c-2f7810a4b4b3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:58.744927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1111/2259] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:44:58.749295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc3b83-7062-4cde-a1c8-1a1d466624e8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:58.759261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc3b83-7062-4cde-a1c8-1a1d466624e8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:58.764353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6ce36-2c73-40c4-a4dc-a517a625abbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:58.765027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6ce36-2c73-40c4-a4dc-a517a625abbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:58.765429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6ce36-2c73-40c4-a4dc-a517a625abbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:58.765712 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6ce36-2c73-40c4-a4dc-a517a625abbd admin admin] PATH_INFO: `/v3/projects/c237fdd93f3e47dea64bd0143f09b819/users/db24ed507ded40fe828c84bc138f893b/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:58.783580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6ce36-2c73-40c4-a4dc-a517a625abbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c237fdd93f3e47dea64bd0143f09b819, user_id=db24ed507ded40fe828c84bc138f893b, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:58.787862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6ce36-2c73-40c4-a4dc-a517a625abbd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:58.882609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1149/2260] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:44:58 2022] PUT /identity/v3/projects/c237fdd93f3e47dea64bd0143f09b819/users/db24ed507ded40fe828c84bc138f893b/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:44:58.887876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7645d52-d12a-4259-b2db-acc4ae1d3b04 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:58.888295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7645d52-d12a-4259-b2db-acc4ae1d3b04 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:58.888662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7645d52-d12a-4259-b2db-acc4ae1d3b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:58.914384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e7645d52-d12a-4259-b2db-acc4ae1d3b04 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:44:58.915610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e7645d52-d12a-4259-b2db-acc4ae1d3b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:58.976670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1112/2261] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:44:58.986941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6ce36-2c73-40c4-a4dc-a517a625abbd admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:58.988732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ddb6ce36-2c73-40c4-a4dc-a517a625abbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:59.015417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:44:59.057386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6ce36-2c73-40c4-a4dc-a517a625abbd admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:59.062046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bf0f3-b599-42c1-8c49-71f169b72e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:59.062990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bf0f3-b599-42c1-8c49-71f169b72e14 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:59.062990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bf0f3-b599-42c1-8c49-71f169b72e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:59.063213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bf0f3-b599-42c1-8c49-71f169b72e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:59.075204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bf0f3-b599-42c1-8c49-71f169b72e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:59.076689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bf0f3-b599-42c1-8c49-71f169b72e14 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:59.087977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1150/2262] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:44:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:44:59.843069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7645d52-d12a-4259-b2db-acc4ae1d3b04 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:44:59.844637 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e7645d52-d12a-4259-b2db-acc4ae1d3b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:44:59.856540 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7645d52-d12a-4259-b2db-acc4ae1d3b04 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:44:59.861185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:44:59.861709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:44:59.861986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:44:59.862301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] PATH_INFO: `/v3/users/93201f697f2448d984a52b8c3958731a` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:44:59.863444 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93201f697f2448d984a52b8c3958731a)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:44:59.865328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:44:59.894091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:45:00.152555 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93201f697f2448d984a52b8c3958731a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:00.153348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93201f697f2448d984a52b8c3958731a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:00.167380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93201f697f2448d984a52b8c3958731a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:00.252422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93201f697f2448d984a52b8c3958731a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:00.253159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93201f697f2448d984a52b8c3958731a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:00.272607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1113/2263] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:44:59 2022] DELETE /identity/v3/users/93201f697f2448d984a52b8c3958731a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:45:00.276756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99394a-c9e0-4a0b-a3ca-555da70e67e6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:00.277251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99394a-c9e0-4a0b-a3ca-555da70e67e6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:00.277493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99394a-c9e0-4a0b-a3ca-555da70e67e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:00.306306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6d99394a-c9e0-4a0b-a3ca-555da70e67e6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:00.307722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6d99394a-c9e0-4a0b-a3ca-555da70e67e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:00.378582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1151/2264] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:45:00.389968 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:00.391393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:00.451828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b47f-1cdf-46fc-9d68-6abd47d5a4f3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:00.455483 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54379c5-2a91-421d-a0b0-9ad48045de9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:00.456088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54379c5-2a91-421d-a0b0-9ad48045de9c service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:00.456363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54379c5-2a91-421d-a0b0-9ad48045de9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:00.456633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54379c5-2a91-421d-a0b0-9ad48045de9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:00.466047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54379c5-2a91-421d-a0b0-9ad48045de9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:00.467497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54379c5-2a91-421d-a0b0-9ad48045de9c service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:00.477644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1114/2265] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:45:01.276570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99394a-c9e0-4a0b-a3ca-555da70e67e6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:01.278073 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6d99394a-c9e0-4a0b-a3ca-555da70e67e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:01.298039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99394a-c9e0-4a0b-a3ca-555da70e67e6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:01.307873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:01.308984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:01.309894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:01.310433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] PATH_INFO: `/v3/projects/47a4c4de23fc4c3894545612c4386ad1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:01.312601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47a4c4de23fc4c3894545612c4386ad1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:01.315497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:01.502912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47a4c4de23fc4c3894545612c4386ad1', 'request_id': 'req-c3083a6f-b2b5-4051-a6e2-583897dc2016'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:01.517684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47a4c4de23fc4c3894545612c4386ad1', 'request_id': 'req-c3083a6f-b2b5-4051-a6e2-583897dc2016'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:01.618684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47a4c4de23fc4c3894545612c4386ad1', 'request_id': 'req-c3083a6f-b2b5-4051-a6e2-583897dc2016'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:01.636835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] The token cache is being invalidate because project 47a4c4de23fc4c3894545612c4386ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:45:01.637142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:01.638927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:01.654957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1152/2266] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:45:01 2022] DELETE /identity/v3/projects/47a4c4de23fc4c3894545612c4386ad1 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:45:01.812906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f185bf-3312-4c51-92c4-d5a2fef6f43b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:01.813838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f185bf-3312-4c51-92c4-d5a2fef6f43b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:01.814306 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f185bf-3312-4c51-92c4-d5a2fef6f43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:01.847724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c2f185bf-3312-4c51-92c4-d5a2fef6f43b None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:01.850357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c2f185bf-3312-4c51-92c4-d5a2fef6f43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:01.914126 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:01.915967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:01.934958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1115/2267] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:45:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:45:01.954457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f185bf-3312-4c51-92c4-d5a2fef6f43b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:01.958024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c2f185bf-3312-4c51-92c4-d5a2fef6f43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:02.031026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:45:02.051885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3083a6f-b2b5-4051-a6e2-583897dc2016 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:02.057868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:02.058463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:02.058743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:02.059217 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] PATH_INFO: `/v3/users/4da5cc3812294213b4a194173b521205` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:02.060395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4da5cc3812294213b4a194173b521205)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:02.062950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:45:02.063322 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:02.094780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f185bf-3312-4c51-92c4-d5a2fef6f43b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:02.101697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686894a3-0739-4aeb-850d-e0d7b9a64b5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:02.104591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686894a3-0739-4aeb-850d-e0d7b9a64b5a service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:02.104591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686894a3-0739-4aeb-850d-e0d7b9a64b5a service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:02.104591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686894a3-0739-4aeb-850d-e0d7b9a64b5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:02.118017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686894a3-0739-4aeb-850d-e0d7b9a64b5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:02.119782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686894a3-0739-4aeb-850d-e0d7b9a64b5a service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:02.137708 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1116/2268] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:45:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 05:45:02.331721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4da5cc3812294213b4a194173b521205'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:02.354505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4da5cc3812294213b4a194173b521205'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:02.358869 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4da5cc3812294213b4a194173b521205'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:02.438241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4da5cc3812294213b4a194173b521205'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:02.439653 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4da5cc3812294213b4a194173b521205'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:02.463419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1153/2269] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:45:01 2022] DELETE /identity/v3/users/4da5cc3812294213b4a194173b521205 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:45:02.468159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b738dad-21b4-46a0-bb48-ddef115736c8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:02.468494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b738dad-21b4-46a0-bb48-ddef115736c8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:02.468846 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b738dad-21b4-46a0-bb48-ddef115736c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:02.494996 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8b738dad-21b4-46a0-bb48-ddef115736c8 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:02.496304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8b738dad-21b4-46a0-bb48-ddef115736c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:02.637990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:02.640493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1117/2270] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:45:02.644061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:02.748539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e952a-3132-4d33-9bfc-cddf28ec8d52 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:02.756515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74900a47-c630-4228-bc17-f9d097458eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:02.761109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900a47-c630-4228-bc17-f9d097458eeb service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:02.761109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900a47-c630-4228-bc17-f9d097458eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:02.761109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900a47-c630-4228-bc17-f9d097458eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:02.776082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900a47-c630-4228-bc17-f9d097458eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:02.779016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900a47-c630-4228-bc17-f9d097458eeb service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:02.798494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1154/2271] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:45:03.770040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b738dad-21b4-46a0-bb48-ddef115736c8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:03.777739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8b738dad-21b4-46a0-bb48-ddef115736c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:03.796275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b738dad-21b4-46a0-bb48-ddef115736c8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:03.803599 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:03.804366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:03.804781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:03.805256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] PATH_INFO: `/v3/projects/3ff186fb007547e2b12bd93a3801f6f1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:03.806536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ff186fb007547e2b12bd93a3801f6f1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:03.808605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:03.977815 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ff186fb007547e2b12bd93a3801f6f1', 'request_id': 'req-8975179e-9f32-49c3-bda3-e88158bbeb92'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:04.000588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ff186fb007547e2b12bd93a3801f6f1', 'request_id': 'req-8975179e-9f32-49c3-bda3-e88158bbeb92'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:04.064322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ff186fb007547e2b12bd93a3801f6f1', 'request_id': 'req-8975179e-9f32-49c3-bda3-e88158bbeb92'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:04.093093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] The token cache is being invalidate because project 3ff186fb007547e2b12bd93a3801f6f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:45:04.093395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:04.094067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:45:04.096195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1118/2272] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:45:03 2022] DELETE /identity/v3/projects/3ff186fb007547e2b12bd93a3801f6f1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:45:07.814333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63960e-1374-4099-87b1-19f455cb47b6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:07.814837 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63960e-1374-4099-87b1-19f455cb47b6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:07.814989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63960e-1374-4099-87b1-19f455cb47b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:07.879476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0a63960e-1374-4099-87b1-19f455cb47b6 None None] MFA Rules not processed for user `db24ed507ded40fe828c84bc138f893b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:07.881104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0a63960e-1374-4099-87b1-19f455cb47b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:07.970724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:45:07.977247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1155/2273] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:45:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:45:07.981856 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:07.983295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:08.080711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975179e-9f32-49c3-bda3-e88158bbeb92 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:08.087481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a75d0-840e-4d62-8b23-95a9b6f942e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:08.088161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a75d0-840e-4d62-8b23-95a9b6f942e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:08.088488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a75d0-840e-4d62-8b23-95a9b6f942e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:08.088765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a75d0-840e-4d62-8b23-95a9b6f942e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:08.089740 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a75d0-840e-4d62-8b23-95a9b6f942e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:08.094545 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a75d0-840e-4d62-8b23-95a9b6f942e1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:08.105860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:45:08.192021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1119/2274] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:45:07 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:45:08.196277 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63960e-1374-4099-87b1-19f455cb47b6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:08.211787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63960e-1374-4099-87b1-19f455cb47b6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:08.297237 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414e9a1-f058-4711-b0dc-5b7a76f04c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:08.297999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414e9a1-f058-4711-b0dc-5b7a76f04c28 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:08.298406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414e9a1-f058-4711-b0dc-5b7a76f04c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:08.298778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414e9a1-f058-4711-b0dc-5b7a76f04c28 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:08.299524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414e9a1-f058-4711-b0dc-5b7a76f04c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:08.301575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414e9a1-f058-4711-b0dc-5b7a76f04c28 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:08.382493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1156/2275] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:45:08 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:45:08.386580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a75d0-840e-4d62-8b23-95a9b6f942e1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:08.402122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a75d0-840e-4d62-8b23-95a9b6f942e1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:08.409021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6989-60a8-4fc4-a795-e7780ccdba8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:08.414297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6989-60a8-4fc4-a795-e7780ccdba8e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:08.414796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6989-60a8-4fc4-a795-e7780ccdba8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:08.415278 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6989-60a8-4fc4-a795-e7780ccdba8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:08.418365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c6989-60a8-4fc4-a795-e7780ccdba8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:08.419939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c6989-60a8-4fc4-a795-e7780ccdba8e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:08.439621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1120/2276] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:45:08.441234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414e9a1-f058-4711-b0dc-5b7a76f04c28 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:08.452299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414e9a1-f058-4711-b0dc-5b7a76f04c28 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:08.457593 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1e4dd-9138-4b1b-87cb-a97b63ed2c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:08.459819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1e4dd-9138-4b1b-87cb-a97b63ed2c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:08.460213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1e4dd-9138-4b1b-87cb-a97b63ed2c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:08.460586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1e4dd-9138-4b1b-87cb-a97b63ed2c36 admin admin] PATH_INFO: `/v3/projects/cd65a0a8255d4f589832dd37a06968c5/users/18e2a2dab2c54d7b81e6fbd0dff0765b/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:08.481161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1e4dd-9138-4b1b-87cb-a97b63ed2c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd65a0a8255d4f589832dd37a06968c5, user_id=18e2a2dab2c54d7b81e6fbd0dff0765b, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:08.490348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1e4dd-9138-4b1b-87cb-a97b63ed2c36 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:08.573781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1157/2277] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:45:08 2022] PUT /identity/v3/projects/cd65a0a8255d4f589832dd37a06968c5/users/18e2a2dab2c54d7b81e6fbd0dff0765b/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:45:08.578025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6989-60a8-4fc4-a795-e7780ccdba8e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:08.589176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6989-60a8-4fc4-a795-e7780ccdba8e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:08.674921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13a950-deda-40a1-ba6f-109dffa18249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:08.675749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a950-deda-40a1-ba6f-109dffa18249 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:08.676208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a950-deda-40a1-ba6f-109dffa18249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:08.676571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13a950-deda-40a1-ba6f-109dffa18249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:08.678641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13a950-deda-40a1-ba6f-109dffa18249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:08.679420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13a950-deda-40a1-ba6f-109dffa18249 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:08.694191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1121/2278] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:45:08.695856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1e4dd-9138-4b1b-87cb-a97b63ed2c36 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:08.707187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1e4dd-9138-4b1b-87cb-a97b63ed2c36 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:08.713288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2c8f3-d460-4ca7-a26b-f56f69bbd36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:08.714021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2c8f3-d460-4ca7-a26b-f56f69bbd36b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:08.714401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2c8f3-d460-4ca7-a26b-f56f69bbd36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:08.714740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2c8f3-d460-4ca7-a26b-f56f69bbd36b admin admin] PATH_INFO: `/v3/projects/cd65a0a8255d4f589832dd37a06968c5/users/18e2a2dab2c54d7b81e6fbd0dff0765b/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:08.716728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2c8f3-d460-4ca7-a26b-f56f69bbd36b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd65a0a8255d4f589832dd37a06968c5, user_id=18e2a2dab2c54d7b81e6fbd0dff0765b, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:08.719666 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2c8f3-d460-4ca7-a26b-f56f69bbd36b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:08.737069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1158/2279] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:45:08 2022] PUT /identity/v3/projects/cd65a0a8255d4f589832dd37a06968c5/users/18e2a2dab2c54d7b81e6fbd0dff0765b/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:45:18.060997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf704e0-326e-4dbe-a389-16a9c77eba6b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:18.061821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf704e0-326e-4dbe-a389-16a9c77eba6b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:18.061821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf704e0-326e-4dbe-a389-16a9c77eba6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:18.114197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ebf704e0-326e-4dbe-a389-16a9c77eba6b None None] MFA Rules not processed for user `18e2a2dab2c54d7b81e6fbd0dff0765b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:18.119721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ebf704e0-326e-4dbe-a389-16a9c77eba6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:18.223208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1122/2280] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:45:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:45:18.241606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2c8f3-d460-4ca7-a26b-f56f69bbd36b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:18.244172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-87f2c8f3-d460-4ca7-a26b-f56f69bbd36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:18.412080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2c8f3-d460-4ca7-a26b-f56f69bbd36b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:18.417239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bc71e-476b-4fec-a361-d8634bd5a617 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34eb8fcef0543c282b3887dfeb74c15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:18.419597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc71e-476b-4fec-a361-d8634bd5a617 service glance] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:18.419597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc71e-476b-4fec-a361-d8634bd5a617 service glance] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:18.419597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bc71e-476b-4fec-a361-d8634bd5a617 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:18.496033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:45:18.517883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bc71e-476b-4fec-a361-d8634bd5a617 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:18.526406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bc71e-476b-4fec-a361-d8634bd5a617 service glance] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:18.530694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1159/2281] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:45:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:45:19.158699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9f9dd-8654-4f0e-be5e-d834fae5a9d4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:19.159577 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9f9dd-8654-4f0e-be5e-d834fae5a9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:19.162354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9f9dd-8654-4f0e-be5e-d834fae5a9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:19.189126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e5e9f9dd-8654-4f0e-be5e-d834fae5a9d4 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:19.190495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e5e9f9dd-8654-4f0e-be5e-d834fae5a9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:19.286254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1123/2282] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:45:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:45:19.300636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bc71e-476b-4fec-a361-d8634bd5a617 service glance] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:19.302278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0e2bc71e-476b-4fec-a361-d8634bd5a617 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:19.401615 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bc71e-476b-4fec-a361-d8634bd5a617 service glance] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:19.413822 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724a8f8-f952-4015-968f-67115500509e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:19.418739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724a8f8-f952-4015-968f-67115500509e service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:19.419122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724a8f8-f952-4015-968f-67115500509e service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:19.419468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724a8f8-f952-4015-968f-67115500509e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:19.432023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724a8f8-f952-4015-968f-67115500509e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:19.433743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724a8f8-f952-4015-968f-67115500509e service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:19.446497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1160/2283] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:45:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:45:28.115187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9125e-b3bc-49c7-9e01-3ae4eb7551d4 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:28.116168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9125e-b3bc-49c7-9e01-3ae4eb7551d4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:28.116590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9125e-b3bc-49c7-9e01-3ae4eb7551d4 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:28.118407 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1124/2284] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:45:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:45:28.127002 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8685733-f324-4087-9614-0f9cfcc6ac0c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:28.127599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8685733-f324-4087-9614-0f9cfcc6ac0c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:28.127599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8685733-f324-4087-9614-0f9cfcc6ac0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:28.168648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c8685733-f324-4087-9614-0f9cfcc6ac0c None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:28.170372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c8685733-f324-4087-9614-0f9cfcc6ac0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:28.386373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1161/2285] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:45:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 04 05:45:28.396888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9125e-b3bc-49c7-9e01-3ae4eb7551d4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:28.399018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-91c9125e-b3bc-49c7-9e01-3ae4eb7551d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:28.409432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:45:28.417020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9125e-b3bc-49c7-9e01-3ae4eb7551d4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:28.421175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969aee3-5a15-47b6-b027-11f26ac62618 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34eb8fcef0543c282b3887dfeb74c15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:28.421882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969aee3-5a15-47b6-b027-11f26ac62618 service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:28.422283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969aee3-5a15-47b6-b027-11f26ac62618 service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:28.423065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969aee3-5a15-47b6-b027-11f26ac62618 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:28.436021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969aee3-5a15-47b6-b027-11f26ac62618 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:28.438333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969aee3-5a15-47b6-b027-11f26ac62618 service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:28.452687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1125/2286] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:45:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:45:28.678819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aeccf7-04be-422f-b2bd-9760ca8a47fa None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:28.679095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aeccf7-04be-422f-b2bd-9760ca8a47fa None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:28.679329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aeccf7-04be-422f-b2bd-9760ca8a47fa None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:28.680910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1162/2287] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:45:28.686703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01334a7b-4860-4331-b6d6-6ebc45b72e8c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:28.687037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01334a7b-4860-4331-b6d6-6ebc45b72e8c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:28.687251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01334a7b-4860-4331-b6d6-6ebc45b72e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:28.739563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-01334a7b-4860-4331-b6d6-6ebc45b72e8c None None] MFA Rules not processed for user `3a0feaf83ee14071806f36d01c36b635`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:28.741254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-01334a7b-4860-4331-b6d6-6ebc45b72e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:28.814586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1126/2288] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:45:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:45:28.821785 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aeccf7-04be-422f-b2bd-9760ca8a47fa None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:28.823715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-64aeccf7-04be-422f-b2bd-9760ca8a47fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:28.842083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aeccf7-04be-422f-b2bd-9760ca8a47fa None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:28.845906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2862b-bf12-4037-b6d7-c897935fc5a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:28.849258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2862b-bf12-4037-b6d7-c897935fc5a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:28.849258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2862b-bf12-4037-b6d7-c897935fc5a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:28.849258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2862b-bf12-4037-b6d7-c897935fc5a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:28.857592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2862b-bf12-4037-b6d7-c897935fc5a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:28.860450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2862b-bf12-4037-b6d7-c897935fc5a1 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:28.905526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1163/2289] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:45:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:45:29.044312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ede53-d780-4778-9044-247c10fb7f39 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:29.044597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ede53-d780-4778-9044-247c10fb7f39 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:29.044877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ede53-d780-4778-9044-247c10fb7f39 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:29.046438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1127/2290] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:45:29.054697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1a959-6e0d-4997-958e-3f515da0244e None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:29.055217 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1a959-6e0d-4997-958e-3f515da0244e None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:29.055485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1a959-6e0d-4997-958e-3f515da0244e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:29.077463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c9a1a959-6e0d-4997-958e-3f515da0244e None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:29.079011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c9a1a959-6e0d-4997-958e-3f515da0244e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:29.089321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1164/2291] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:45:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:45:29.100970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ede53-d780-4778-9044-247c10fb7f39 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:29.103056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-364ede53-d780-4778-9044-247c10fb7f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:29.121023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ede53-d780-4778-9044-247c10fb7f39 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:29.125390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d1e64-05fe-40e1-91e6-8cf402dbf3a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:29.126011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d1e64-05fe-40e1-91e6-8cf402dbf3a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:29.126329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d1e64-05fe-40e1-91e6-8cf402dbf3a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:29.126611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d1e64-05fe-40e1-91e6-8cf402dbf3a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:29.138655 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d1e64-05fe-40e1-91e6-8cf402dbf3a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:29.140164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d1e64-05fe-40e1-91e6-8cf402dbf3a6 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:29.152651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1128/2292] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:45:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:45:32.275286 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a46347-f913-4a93-b641-234737fce266 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:32.275832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a46347-f913-4a93-b641-234737fce266 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:32.275832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a46347-f913-4a93-b641-234737fce266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:32.294198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-50a46347-f913-4a93-b641-234737fce266 None None] MFA Rules not processed for user `db24ed507ded40fe828c84bc138f893b`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:32.295380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-50a46347-f913-4a93-b641-234737fce266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:32.302978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1165/2293] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:45:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:45:39.146571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=34 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:45:39.764690 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274cdfd-fda8-4416-b33e-105523ae5d44 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:39.764982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274cdfd-fda8-4416-b33e-105523ae5d44 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:39.765262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274cdfd-fda8-4416-b33e-105523ae5d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:39.788355 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5274cdfd-fda8-4416-b33e-105523ae5d44 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:39.789685 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5274cdfd-fda8-4416-b33e-105523ae5d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:39.797536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1129/2294] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:45:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:45:39.812036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a46347-f913-4a93-b641-234737fce266 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:39.823901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a46347-f913-4a93-b641-234737fce266 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:39.829129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1a096-75f5-4160-814e-e6ef6fb87fdd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:39.829923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1a096-75f5-4160-814e-e6ef6fb87fdd service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:39.830242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1a096-75f5-4160-814e-e6ef6fb87fdd service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:39.830862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1a096-75f5-4160-814e-e6ef6fb87fdd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:39.840622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1a096-75f5-4160-814e-e6ef6fb87fdd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:39.842233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1a096-75f5-4160-814e-e6ef6fb87fdd service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:39.851801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1166/2295] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:45:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:45:39.956207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2a829-8524-42f9-a5b8-c8f79e9246f6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:39.956207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2a829-8524-42f9-a5b8-c8f79e9246f6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:39.956207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2a829-8524-42f9-a5b8-c8f79e9246f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:39.978993 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-08c2a829-8524-42f9-a5b8-c8f79e9246f6 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:39.980258 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-08c2a829-8524-42f9-a5b8-c8f79e9246f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:39.988439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1130/2296] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:45:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:45:40.002301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1a096-75f5-4160-814e-e6ef6fb87fdd service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:45:40.012840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1a096-75f5-4160-814e-e6ef6fb87fdd service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:45:40.018434 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17a118-5d3a-48df-9093-ff5a67d543b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:45:40.019037 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17a118-5d3a-48df-9093-ff5a67d543b9 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:40.019384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17a118-5d3a-48df-9093-ff5a67d543b9 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:40.019660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17a118-5d3a-48df-9093-ff5a67d543b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:40.028576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17a118-5d3a-48df-9093-ff5a67d543b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:45:40.030157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17a118-5d3a-48df-9093-ff5a67d543b9 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:45:40.043523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1167/2297] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:45:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:45:45.573775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e75dab-6f6a-4708-8ffe-500d0414353e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:45:45.574406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e75dab-6f6a-4708-8ffe-500d0414353e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:45:45.574406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e75dab-6f6a-4708-8ffe-500d0414353e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:45:45.594311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-07e75dab-6f6a-4708-8ffe-500d0414353e None None] MFA Rules not processed for user `877575c9e0264241b9eab868a89cbbe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:45:45.595549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-07e75dab-6f6a-4708-8ffe-500d0414353e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:45:45.656141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1131/2298] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:45:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:45:50.034830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:45:55.649134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=31 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:46:00.334433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17a118-5d3a-48df-9093-ff5a67d543b9 service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:46:00.336732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9e17a118-5d3a-48df-9093-ff5a67d543b9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:46:00.417285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17a118-5d3a-48df-9093-ff5a67d543b9 service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:46:00.423441 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd2e0b-cd3c-4f5d-9ef9-1aeb56858d42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:46:00.424143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd2e0b-cd3c-4f5d-9ef9-1aeb56858d42 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:46:00.424426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd2e0b-cd3c-4f5d-9ef9-1aeb56858d42 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:46:00.424869 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd2e0b-cd3c-4f5d-9ef9-1aeb56858d42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:46:00.427309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-33bd2e0b-cd3c-4f5d-9ef9-1aeb56858d42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:46:00.440183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd2e0b-cd3c-4f5d-9ef9-1aeb56858d42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:46:00.441923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd2e0b-cd3c-4f5d-9ef9-1aeb56858d42 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:46:00.453031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1168/2299] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:46:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:46:10.448988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=32 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:46:17.491912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e32711-749a-4796-b19c-83e218e59212 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:46:17.492533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e32711-749a-4796-b19c-83e218e59212 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:46:17.492975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e32711-749a-4796-b19c-83e218e59212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:46:17.520211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c6e32711-749a-4796-b19c-83e218e59212 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:46:17.524447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c6e32711-749a-4796-b19c-83e218e59212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:46:17.529359 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1132/2300] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 05:46:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:46:17.543188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd2e0b-cd3c-4f5d-9ef9-1aeb56858d42 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:46:17.555057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd2e0b-cd3c-4f5d-9ef9-1aeb56858d42 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:46:17.560642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fada74-cb05-4f75-8f3c-f7422da8c447 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:46:17.561502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fada74-cb05-4f75-8f3c-f7422da8c447 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:46:17.561780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fada74-cb05-4f75-8f3c-f7422da8c447 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:46:17.562100 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fada74-cb05-4f75-8f3c-f7422da8c447 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:46:17.571245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fada74-cb05-4f75-8f3c-f7422da8c447 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:46:17.572805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fada74-cb05-4f75-8f3c-f7422da8c447 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:46:17.581345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1169/2301] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:46:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:46:18.141752 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbdafa-c0c8-4fa1-99ac-874d558aae26 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:46:18.142039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbdafa-c0c8-4fa1-99ac-874d558aae26 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:46:18.142338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbdafa-c0c8-4fa1-99ac-874d558aae26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:46:18.164318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-64bbdafa-c0c8-4fa1-99ac-874d558aae26 None None] MFA Rules not processed for user `c1b48a973c20470a9af23c77e3da5b72`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:46:18.165970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-64bbdafa-c0c8-4fa1-99ac-874d558aae26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:46:18.232713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1133/2302] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:46:27.578448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:46:28.224923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=28 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:46:28.841341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68470730-68c0-4c7a-b78f-6a4550f95bbd None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:46:28.841883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68470730-68c0-4c7a-b78f-6a4550f95bbd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:46:28.841883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68470730-68c0-4c7a-b78f-6a4550f95bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:46:28.861809 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-68470730-68c0-4c7a-b78f-6a4550f95bbd None None] MFA Rules not processed for user `c66c7ddbf18d4f1fb8c890f548031f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:46:28.863050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-68470730-68c0-4c7a-b78f-6a4550f95bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:46:28.923717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1170/2303] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:46:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:46:31.131550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbdafa-c0c8-4fa1-99ac-874d558aae26 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:46:31.149091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbdafa-c0c8-4fa1-99ac-874d558aae26 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:46:31.153387 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f9ce2-c888-402a-9883-597249095c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:46:31.154427 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f9ce2-c888-402a-9883-597249095c5e service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:46:31.155492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f9ce2-c888-402a-9883-597249095c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:46:31.156102 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f9ce2-c888-402a-9883-597249095c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:46:31.173401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f9ce2-c888-402a-9883-597249095c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:46:31.175316 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f9ce2-c888-402a-9883-597249095c5e service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:46:31.187765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1134/2304] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:46:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:46:38.916710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=25 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:46:41.182456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:47:25.459556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68470730-68c0-4c7a-b78f-6a4550f95bbd None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:25.463371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-68470730-68c0-4c7a-b78f-6a4550f95bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:25.541683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68470730-68c0-4c7a-b78f-6a4550f95bbd None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:25.547054 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:25.547698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:25.547973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:25.548245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] PATH_INFO: `/v3/users/877575c9e0264241b9eab868a89cbbe8` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:25.549373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=877575c9e0264241b9eab868a89cbbe8)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:25.551332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:25.732968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '877575c9e0264241b9eab868a89cbbe8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:47:25.746243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '877575c9e0264241b9eab868a89cbbe8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:47:25.747250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '877575c9e0264241b9eab868a89cbbe8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:47:25.815668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '877575c9e0264241b9eab868a89cbbe8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:47:25.816327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-81993046-f442-4430-88c3-9f0b4a055517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '877575c9e0264241b9eab868a89cbbe8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:47:25.830663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1171/2305] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:47:25 2022] DELETE /identity/v3/users/877575c9e0264241b9eab868a89cbbe8 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:47:26.063500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f9ce2-c888-402a-9883-597249095c5e service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:26.075082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f9ce2-c888-402a-9883-597249095c5e service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:26.141358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:26.142524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:26.142524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:26.142851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] PATH_INFO: `/v3/projects/e003d2bea5f04ac7a70fd2b7d2ee1f91` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:26.144071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e003d2bea5f04ac7a70fd2b7d2ee1f91)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:26.146118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:26.259957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e003d2bea5f04ac7a70fd2b7d2ee1f91', 'request_id': 'req-bfcb7716-39ca-4a2c-948a-097bcae97986'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:47:26.267051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e003d2bea5f04ac7a70fd2b7d2ee1f91', 'request_id': 'req-bfcb7716-39ca-4a2c-948a-097bcae97986'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:47:26.350477 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e003d2bea5f04ac7a70fd2b7d2ee1f91', 'request_id': 'req-bfcb7716-39ca-4a2c-948a-097bcae97986'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:47:26.361818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] The token cache is being invalidate because project e003d2bea5f04ac7a70fd2b7d2ee1f91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:47:26.362132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:47:26.362769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bfcb7716-39ca-4a2c-948a-097bcae97986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:47:26.364246 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1135/2306] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:47:26 2022] DELETE /identity/v3/projects/e003d2bea5f04ac7a70fd2b7d2ee1f91 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:47:26.370869 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea03b-99e5-4fa5-9d8f-299ccf9d58b8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:26.371206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea03b-99e5-4fa5-9d8f-299ccf9d58b8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:26.371435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea03b-99e5-4fa5-9d8f-299ccf9d58b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:26.391543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-536ea03b-99e5-4fa5-9d8f-299ccf9d58b8 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:47:26.392744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-536ea03b-99e5-4fa5-9d8f-299ccf9d58b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:26.452784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1172/2307] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:47:26.461746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494df70-472e-482b-8a42-e30835cb5a0c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:26.462707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494df70-472e-482b-8a42-e30835cb5a0c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:26.462707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494df70-472e-482b-8a42-e30835cb5a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:26.490213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3494df70-472e-482b-8a42-e30835cb5a0c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:47:26.491473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3494df70-472e-482b-8a42-e30835cb5a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:26.500058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1136/2308] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:47:26.504454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea03b-99e5-4fa5-9d8f-299ccf9d58b8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:26.514263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea03b-99e5-4fa5-9d8f-299ccf9d58b8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:26.518905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb24cf6-f0c5-4937-8864-11bb0f457d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:26.519504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb24cf6-f0c5-4937-8864-11bb0f457d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:26.519785 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb24cf6-f0c5-4937-8864-11bb0f457d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:26.520058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb24cf6-f0c5-4937-8864-11bb0f457d9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:26.520730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb24cf6-f0c5-4937-8864-11bb0f457d9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:47:26.521032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb24cf6-f0c5-4937-8864-11bb0f457d9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:26.522674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb24cf6-f0c5-4937-8864-11bb0f457d9d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:26.534327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1173/2309] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:47:26.538539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494df70-472e-482b-8a42-e30835cb5a0c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:26.548292 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494df70-472e-482b-8a42-e30835cb5a0c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:26.556488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad215e26-0d70-420c-850c-2a27027caa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:26.557123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad215e26-0d70-420c-850c-2a27027caa91 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:26.557403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad215e26-0d70-420c-850c-2a27027caa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:26.557677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad215e26-0d70-420c-850c-2a27027caa91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:26.558356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad215e26-0d70-420c-850c-2a27027caa91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:26.560243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad215e26-0d70-420c-850c-2a27027caa91 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:26.661283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1137/2310] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:47:26 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:47:26.665179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb24cf6-f0c5-4937-8864-11bb0f457d9d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:26.675819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb24cf6-f0c5-4937-8864-11bb0f457d9d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:26.679551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:47:26.732355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29deba1d-3a9a-4c13-9dca-4f414e4b73ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:26.733610 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29deba1d-3a9a-4c13-9dca-4f414e4b73ff admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:26.733885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29deba1d-3a9a-4c13-9dca-4f414e4b73ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:26.734230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29deba1d-3a9a-4c13-9dca-4f414e4b73ff admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:26.734807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29deba1d-3a9a-4c13-9dca-4f414e4b73ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:26.736718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29deba1d-3a9a-4c13-9dca-4f414e4b73ff admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:26.867847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1174/2311] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:47:26 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:47:26.871144 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad215e26-0d70-420c-850c-2a27027caa91 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:26.881425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad215e26-0d70-420c-850c-2a27027caa91 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:26.886495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7f83f-c7ce-48b7-afae-1048ee9701f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:26.887110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7f83f-c7ce-48b7-afae-1048ee9701f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:26.887385 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7f83f-c7ce-48b7-afae-1048ee9701f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:26.887654 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7f83f-c7ce-48b7-afae-1048ee9701f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:26.888564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7f83f-c7ce-48b7-afae-1048ee9701f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:26.893211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7f83f-c7ce-48b7-afae-1048ee9701f1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:26.902698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1138/2312] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:47:26.906107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29deba1d-3a9a-4c13-9dca-4f414e4b73ff admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:26.916673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29deba1d-3a9a-4c13-9dca-4f414e4b73ff admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:26.922332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6b8b7-bce5-4f64-b1bd-20103c70c4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:26.923323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6b8b7-bce5-4f64-b1bd-20103c70c4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:26.923917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6b8b7-bce5-4f64-b1bd-20103c70c4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:26.924787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6b8b7-bce5-4f64-b1bd-20103c70c4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:26.925435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6b8b7-bce5-4f64-b1bd-20103c70c4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:26.927062 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6b8b7-bce5-4f64-b1bd-20103c70c4ad admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:26.937918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1175/2313] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:47:26.942363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7f83f-c7ce-48b7-afae-1048ee9701f1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:26.952287 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7f83f-c7ce-48b7-afae-1048ee9701f1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:26.956745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56f2d4-3e4a-4063-bff3-2be0d843b0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:26.957391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56f2d4-3e4a-4063-bff3-2be0d843b0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:26.957667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56f2d4-3e4a-4063-bff3-2be0d843b0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:26.957936 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56f2d4-3e4a-4063-bff3-2be0d843b0dd admin admin] PATH_INFO: `/v3/projects/45a5869786d843fe9fb5cdb6b3596c2f/users/0c0c0a25340a451b970cc3de8bc05806/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:26.973049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56f2d4-3e4a-4063-bff3-2be0d843b0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a5869786d843fe9fb5cdb6b3596c2f, user_id=0c0c0a25340a451b970cc3de8bc05806, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:26.975817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56f2d4-3e4a-4063-bff3-2be0d843b0dd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:27.048751 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1139/2314] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:47:26 2022] PUT /identity/v3/projects/45a5869786d843fe9fb5cdb6b3596c2f/users/0c0c0a25340a451b970cc3de8bc05806/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:47:27.053911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fff1a-5eff-4670-a19e-437d8d734566 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:27.054222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fff1a-5eff-4670-a19e-437d8d734566 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:27.054503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fff1a-5eff-4670-a19e-437d8d734566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:27.074906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-345fff1a-5eff-4670-a19e-437d8d734566 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:47:27.076127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-345fff1a-5eff-4670-a19e-437d8d734566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:27.143453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1176/2315] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:47:27.153444 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56f2d4-3e4a-4063-bff3-2be0d843b0dd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:27.155837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aa56f2d4-3e4a-4063-bff3-2be0d843b0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:27.186513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:47:27.213283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56f2d4-3e4a-4063-bff3-2be0d843b0dd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:27.217664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8395ec-2a56-4135-9203-15ed8644bb96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:27.218448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8395ec-2a56-4135-9203-15ed8644bb96 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:27.218589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8395ec-2a56-4135-9203-15ed8644bb96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:27.218855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8395ec-2a56-4135-9203-15ed8644bb96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:27.229436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8395ec-2a56-4135-9203-15ed8644bb96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:27.229734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8395ec-2a56-4135-9203-15ed8644bb96 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:27.239291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1140/2316] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:47:30.085308 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367dbbe-bfab-4da1-9e02-f6341c958cfd None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:30.085871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367dbbe-bfab-4da1-9e02-f6341c958cfd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:30.085871 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367dbbe-bfab-4da1-9e02-f6341c958cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:30.104912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5367dbbe-bfab-4da1-9e02-f6341c958cfd None None] MFA Rules not processed for user `6f1635c9c6cf4fb092c0f03a02e36174`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:47:30.106091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5367dbbe-bfab-4da1-9e02-f6341c958cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:30.176568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1177/2317] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:47:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:47:34.815822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809952d-3c25-44e4-ae7d-5876da6178a2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:34.816403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809952d-3c25-44e4-ae7d-5876da6178a2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:34.816403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809952d-3c25-44e4-ae7d-5876da6178a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:34.899044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-8809952d-3c25-44e4-ae7d-5876da6178a2 None None] MFA Rules not processed for user `0c0c0a25340a451b970cc3de8bc05806`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:47:34.901193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8809952d-3c25-44e4-ae7d-5876da6178a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:35.027209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1141/2318] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:47:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:47:35.032278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367dbbe-bfab-4da1-9e02-f6341c958cfd None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:35.046270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367dbbe-bfab-4da1-9e02-f6341c958cfd None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:35.050901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060e5c1-53f5-4f1f-a197-0deda0aa3f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:35.051516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060e5c1-53f5-4f1f-a197-0deda0aa3f22 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:35.052372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060e5c1-53f5-4f1f-a197-0deda0aa3f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:35.052653 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060e5c1-53f5-4f1f-a197-0deda0aa3f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:35.053325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060e5c1-53f5-4f1f-a197-0deda0aa3f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:35.055397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060e5c1-53f5-4f1f-a197-0deda0aa3f22 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:35.165454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809952d-3c25-44e4-ae7d-5876da6178a2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:35.165965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1178/2319] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:47:35 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:47:35.181629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809952d-3c25-44e4-ae7d-5876da6178a2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:35.250486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90432a0e-80cd-4f74-9e04-07bd7af098c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:35.252700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90432a0e-80cd-4f74-9e04-07bd7af098c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:35.253117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90432a0e-80cd-4f74-9e04-07bd7af098c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:35.253553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90432a0e-80cd-4f74-9e04-07bd7af098c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:35.254445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90432a0e-80cd-4f74-9e04-07bd7af098c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:35.256574 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90432a0e-80cd-4f74-9e04-07bd7af098c1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:35.359444 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1142/2320] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:47:35 2022] POST /identity/v3/users => generated 295 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:47:35.359670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060e5c1-53f5-4f1f-a197-0deda0aa3f22 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:35.378700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060e5c1-53f5-4f1f-a197-0deda0aa3f22 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:35.384899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac15d6-4e82-4f38-b97a-1760b68a9fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:35.385646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac15d6-4e82-4f38-b97a-1760b68a9fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:35.385921 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac15d6-4e82-4f38-b97a-1760b68a9fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:35.386399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac15d6-4e82-4f38-b97a-1760b68a9fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:35.392438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac15d6-4e82-4f38-b97a-1760b68a9fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:35.392438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac15d6-4e82-4f38-b97a-1760b68a9fc8 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:35.392438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:47:35.416849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1179/2321] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:47:35.422097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90432a0e-80cd-4f74-9e04-07bd7af098c1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:35.436596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90432a0e-80cd-4f74-9e04-07bd7af098c1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:35.453023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2083d9f-b8b5-487a-a434-f4f3b9f96aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:35.453707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2083d9f-b8b5-487a-a434-f4f3b9f96aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:35.453987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2083d9f-b8b5-487a-a434-f4f3b9f96aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:35.454305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2083d9f-b8b5-487a-a434-f4f3b9f96aa5 admin admin] PATH_INFO: `/v3/projects/b1c0de68f6474ce2b0ec27fd91a5a166/users/3e47eb113db74cc0a40906cca4d2f403/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:35.488207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2083d9f-b8b5-487a-a434-f4f3b9f96aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c0de68f6474ce2b0ec27fd91a5a166, user_id=3e47eb113db74cc0a40906cca4d2f403, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:35.491543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2083d9f-b8b5-487a-a434-f4f3b9f96aa5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:35.589782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1143/2322] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:47:35 2022] PUT /identity/v3/projects/b1c0de68f6474ce2b0ec27fd91a5a166/users/3e47eb113db74cc0a40906cca4d2f403/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:47:35.594121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac15d6-4e82-4f38-b97a-1760b68a9fc8 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:35.606538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac15d6-4e82-4f38-b97a-1760b68a9fc8 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:35.701066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bba54e-3a79-46e4-8f15-ac228e1bdb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:35.702237 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bba54e-3a79-46e4-8f15-ac228e1bdb43 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:35.702613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bba54e-3a79-46e4-8f15-ac228e1bdb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:35.702984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bba54e-3a79-46e4-8f15-ac228e1bdb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:35.703991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bba54e-3a79-46e4-8f15-ac228e1bdb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:35.718085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bba54e-3a79-46e4-8f15-ac228e1bdb43 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:35.742803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1180/2323] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:47:35.742943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2083d9f-b8b5-487a-a434-f4f3b9f96aa5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:35.758982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2083d9f-b8b5-487a-a434-f4f3b9f96aa5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:35.775432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03784d0-8be2-4c91-87e5-ebc9de816dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:35.775432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03784d0-8be2-4c91-87e5-ebc9de816dc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:35.775432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03784d0-8be2-4c91-87e5-ebc9de816dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:35.775432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03784d0-8be2-4c91-87e5-ebc9de816dc7 admin admin] PATH_INFO: `/v3/projects/b1c0de68f6474ce2b0ec27fd91a5a166/users/3e47eb113db74cc0a40906cca4d2f403/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:35.775885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03784d0-8be2-4c91-87e5-ebc9de816dc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c0de68f6474ce2b0ec27fd91a5a166, user_id=3e47eb113db74cc0a40906cca4d2f403, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:35.775885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03784d0-8be2-4c91-87e5-ebc9de816dc7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:35.798741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1144/2324] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:47:35 2022] PUT /identity/v3/projects/b1c0de68f6474ce2b0ec27fd91a5a166/users/3e47eb113db74cc0a40906cca4d2f403/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:47:41.766642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bba54e-3a79-46e4-8f15-ac228e1bdb43 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:41.770307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-04bba54e-3a79-46e4-8f15-ac228e1bdb43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:41.874701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bba54e-3a79-46e4-8f15-ac228e1bdb43 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:41.969805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5c200-507a-46ae-bd00-def6debd9e52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:41.970809 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5c200-507a-46ae-bd00-def6debd9e52 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:41.971165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5c200-507a-46ae-bd00-def6debd9e52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:41.971513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5c200-507a-46ae-bd00-def6debd9e52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:41.973846 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-86e5c200-507a-46ae-bd00-def6debd9e52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:41.993891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76657cc-c551-4f34-b470-9136292371d3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:41.994873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76657cc-c551-4f34-b470-9136292371d3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:41.995343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76657cc-c551-4f34-b470-9136292371d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:42.023341 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b76657cc-c551-4f34-b470-9136292371d3 None None] MFA Rules not processed for user `537981b1b3a14cb4b10f336ef0362358`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:47:42.028798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b76657cc-c551-4f34-b470-9136292371d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:42.037393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:47:42.068897 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:47:42.092607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5c200-507a-46ae-bd00-def6debd9e52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:42.096423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5c200-507a-46ae-bd00-def6debd9e52 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:42.129660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1181/2325] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:47:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:47:42.130572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1145/2326] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:47:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:47:43.329531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c792008-332d-4068-bfc9-6cf5f12b85f0 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:43.330040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c792008-332d-4068-bfc9-6cf5f12b85f0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:43.330131 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c792008-332d-4068-bfc9-6cf5f12b85f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:43.365958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9c792008-332d-4068-bfc9-6cf5f12b85f0 None None] MFA Rules not processed for user `3e47eb113db74cc0a40906cca4d2f403`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:47:43.367290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9c792008-332d-4068-bfc9-6cf5f12b85f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:43.458751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1182/2327] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:47:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:47:43.465078 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76657cc-c551-4f34-b470-9136292371d3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:43.467272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b76657cc-c551-4f34-b470-9136292371d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:43.495145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76657cc-c551-4f34-b470-9136292371d3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:43.501554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d7555-3156-44b0-b870-c7100c45d501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:43.503183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d7555-3156-44b0-b870-c7100c45d501 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:43.503183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d7555-3156-44b0-b870-c7100c45d501 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:43.503183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d7555-3156-44b0-b870-c7100c45d501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:43.567160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d7555-3156-44b0-b870-c7100c45d501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:43.568741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d7555-3156-44b0-b870-c7100c45d501 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:43.579534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1146/2328] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:47:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:47:45.554883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c792008-332d-4068-bfc9-6cf5f12b85f0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:45.563645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9c792008-332d-4068-bfc9-6cf5f12b85f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:45.668124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c792008-332d-4068-bfc9-6cf5f12b85f0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:45.676655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76505e5-adaf-4d33-be6d-5cf5407f6868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:45.682521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76505e5-adaf-4d33-be6d-5cf5407f6868 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:45.682521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76505e5-adaf-4d33-be6d-5cf5407f6868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:45.682521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76505e5-adaf-4d33-be6d-5cf5407f6868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:45.682521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b76505e5-adaf-4d33-be6d-5cf5407f6868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:45.699588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76505e5-adaf-4d33-be6d-5cf5407f6868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:45.702029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76505e5-adaf-4d33-be6d-5cf5407f6868 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:45.729875 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1183/2329] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:47:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:47:50.313589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d7555-3156-44b0-b870-c7100c45d501 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:50.323916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d7555-3156-44b0-b870-c7100c45d501 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:50.327765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff4b41-9ee3-4cbd-9995-f953535baddb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:50.328414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff4b41-9ee3-4cbd-9995-f953535baddb service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:50.328693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff4b41-9ee3-4cbd-9995-f953535baddb service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:50.328967 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff4b41-9ee3-4cbd-9995-f953535baddb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:50.330924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bfff4b41-9ee3-4cbd-9995-f953535baddb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:50.351968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76505e5-adaf-4d33-be6d-5cf5407f6868 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:50.353985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b76505e5-adaf-4d33-be6d-5cf5407f6868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:50.379532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76505e5-adaf-4d33-be6d-5cf5407f6868 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:50.391484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85cd64-b6c4-4533-8154-9bed04e19f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:50.392470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85cd64-b6c4-4533-8154-9bed04e19f71 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:50.392752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85cd64-b6c4-4533-8154-9bed04e19f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:50.393031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85cd64-b6c4-4533-8154-9bed04e19f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:50.395259 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6f85cd64-b6c4-4533-8154-9bed04e19f71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:50.405586 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff4b41-9ee3-4cbd-9995-f953535baddb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:50.407742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff4b41-9ee3-4cbd-9995-f953535baddb service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:50.416684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85cd64-b6c4-4533-8154-9bed04e19f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:50.418341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85cd64-b6c4-4533-8154-9bed04e19f71 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:50.423419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1147/2330] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:47:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:47:50.443936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1184/2331] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:47:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:47:57.317517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff4b41-9ee3-4cbd-9995-f953535baddb service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:47:57.330228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff4b41-9ee3-4cbd-9995-f953535baddb service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:47:57.335778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2c97c-2763-4011-9ab2-1d69f03da583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:47:57.336387 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2c97c-2763-4011-9ab2-1d69f03da583 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:47:57.336669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2c97c-2763-4011-9ab2-1d69f03da583 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:47:57.336941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2c97c-2763-4011-9ab2-1d69f03da583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:47:57.338693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e5b2c97c-2763-4011-9ab2-1d69f03da583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:47:57.350729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2c97c-2763-4011-9ab2-1d69f03da583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:47:57.352558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2c97c-2763-4011-9ab2-1d69f03da583 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:47:57.366069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1148/2332] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:47:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:48:00.425103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=70 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:48:03.566657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad69fe-6cb0-4d6a-a341-fd0fa2a306b3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:03.567309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad69fe-6cb0-4d6a-a341-fd0fa2a306b3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:03.567309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad69fe-6cb0-4d6a-a341-fd0fa2a306b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:03.586781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-70ad69fe-6cb0-4d6a-a341-fd0fa2a306b3 None None] MFA Rules not processed for user `0c0c0a25340a451b970cc3de8bc05806`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:48:03.587993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-70ad69fe-6cb0-4d6a-a341-fd0fa2a306b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:03.595021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1185/2333] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:48:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:48:03.914033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2c97c-2763-4011-9ab2-1d69f03da583 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:03.923903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2c97c-2763-4011-9ab2-1d69f03da583 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:03.927650 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1622417-93ce-4d53-8715-40ff92b5ceab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:03.928244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1622417-93ce-4d53-8715-40ff92b5ceab service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:03.928548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1622417-93ce-4d53-8715-40ff92b5ceab service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:03.928816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1622417-93ce-4d53-8715-40ff92b5ceab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:03.930606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e1622417-93ce-4d53-8715-40ff92b5ceab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:03.940777 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1622417-93ce-4d53-8715-40ff92b5ceab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:03.942192 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1622417-93ce-4d53-8715-40ff92b5ceab service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:03.949957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1149/2334] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:48:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:48:04.136095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efda09-3c45-48f2-a205-fe32d3291efa None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:04.136522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efda09-3c45-48f2-a205-fe32d3291efa None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:04.136937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efda09-3c45-48f2-a205-fe32d3291efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:04.156309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-07efda09-3c45-48f2-a205-fe32d3291efa None None] MFA Rules not processed for user `c66c7ddbf18d4f1fb8c890f548031f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:48:04.157531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-07efda09-3c45-48f2-a205-fe32d3291efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:04.165433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1186/2335] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:48:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:48:04.470571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1622417-93ce-4d53-8715-40ff92b5ceab service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:04.479176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1622417-93ce-4d53-8715-40ff92b5ceab service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:04.483713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebf06f-0fa0-4ad7-8e0d-2efd9a0334a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:04.484317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebf06f-0fa0-4ad7-8e0d-2efd9a0334a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:04.484600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebf06f-0fa0-4ad7-8e0d-2efd9a0334a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:04.484870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebf06f-0fa0-4ad7-8e0d-2efd9a0334a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:04.486627 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-24ebf06f-0fa0-4ad7-8e0d-2efd9a0334a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:04.496288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebf06f-0fa0-4ad7-8e0d-2efd9a0334a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:04.497696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebf06f-0fa0-4ad7-8e0d-2efd9a0334a2 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:04.506280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1150/2336] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:48:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:48:04.719674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad5767-94d8-42b2-88bc-50753e75ddad None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:04.720537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad5767-94d8-42b2-88bc-50753e75ddad None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:04.721268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad5767-94d8-42b2-88bc-50753e75ddad None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:04.723507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1187/2337] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 05:48:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:48:04.729604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb5c8-4d9d-495b-9a8d-7f80048684ed None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:04.729966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb5c8-4d9d-495b-9a8d-7f80048684ed None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:04.730351 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bb5c8-4d9d-495b-9a8d-7f80048684ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:04.764315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-140bb5c8-4d9d-495b-9a8d-7f80048684ed None None] MFA Rules not processed for user `e95901cce5d448acb2571976163868a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:48:04.765662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-140bb5c8-4d9d-495b-9a8d-7f80048684ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:04.775839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:48:04.836317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1151/2338] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 05:48:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:48:04.854695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad5767-94d8-42b2-88bc-50753e75ddad None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:04.869176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad5767-94d8-42b2-88bc-50753e75ddad None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:04.874437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c9ef8-5094-4246-900f-843306124b55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:04.875150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c9ef8-5094-4246-900f-843306124b55 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:04.875683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c9ef8-5094-4246-900f-843306124b55 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:04.876023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c9ef8-5094-4246-900f-843306124b55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:04.877838 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-745c9ef8-5094-4246-900f-843306124b55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:04.890686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c9ef8-5094-4246-900f-843306124b55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:04.892554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c9ef8-5094-4246-900f-843306124b55 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:04.908890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1188/2339] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:48:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:48:08.420182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c598013-2ca5-45b7-b711-e3cf6971ca9b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:08.420729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c598013-2ca5-45b7-b711-e3cf6971ca9b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:08.420900 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c598013-2ca5-45b7-b711-e3cf6971ca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:08.440070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4c598013-2ca5-45b7-b711-e3cf6971ca9b None None] MFA Rules not processed for user `c1b48a973c20470a9af23c77e3da5b72`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:48:08.441309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4c598013-2ca5-45b7-b711-e3cf6971ca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:08.495942 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1152/2340] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:48:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:48:08.985010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c9ef8-5094-4246-900f-843306124b55 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:08.998805 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c9ef8-5094-4246-900f-843306124b55 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:09.000506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce558ae0-beeb-43ec-bee6-0d7e8248f02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:09.001187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce558ae0-beeb-43ec-bee6-0d7e8248f02d service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:09.001467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce558ae0-beeb-43ec-bee6-0d7e8248f02d service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:09.001741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce558ae0-beeb-43ec-bee6-0d7e8248f02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:09.003701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ce558ae0-beeb-43ec-bee6-0d7e8248f02d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:09.026180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce558ae0-beeb-43ec-bee6-0d7e8248f02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:09.027882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce558ae0-beeb-43ec-bee6-0d7e8248f02d service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:09.041096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1189/2341] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:48:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:48:11.619723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c598013-2ca5-45b7-b711-e3cf6971ca9b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:11.634053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c598013-2ca5-45b7-b711-e3cf6971ca9b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:11.641023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4272423-72c6-4d6e-a55a-d12dba1c2955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:11.641914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4272423-72c6-4d6e-a55a-d12dba1c2955 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:11.642529 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4272423-72c6-4d6e-a55a-d12dba1c2955 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:11.643010 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4272423-72c6-4d6e-a55a-d12dba1c2955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:11.645091 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d4272423-72c6-4d6e-a55a-d12dba1c2955 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:11.660353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4272423-72c6-4d6e-a55a-d12dba1c2955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:11.662309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4272423-72c6-4d6e-a55a-d12dba1c2955 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:11.673999 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1153/2342] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:48:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:48:15.601800 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce558ae0-beeb-43ec-bee6-0d7e8248f02d service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:15.603563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ce558ae0-beeb-43ec-bee6-0d7e8248f02d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:15.621632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce558ae0-beeb-43ec-bee6-0d7e8248f02d service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:15.627426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839df148-81cd-46e2-af1d-3bfefb1f4976 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:15.628044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df148-81cd-46e2-af1d-3bfefb1f4976 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:15.629722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df148-81cd-46e2-af1d-3bfefb1f4976 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:15.629722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839df148-81cd-46e2-af1d-3bfefb1f4976 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:15.630389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-839df148-81cd-46e2-af1d-3bfefb1f4976 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:15.644504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df148-81cd-46e2-af1d-3bfefb1f4976 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:15.646127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839df148-81cd-46e2-af1d-3bfefb1f4976 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:15.657228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1190/2343] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:48:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:48:21.669748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=54 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:48:25.652241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=24 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:48:27.960058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1920c7-68cc-4294-af0e-281531c36be5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:27.960883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1920c7-68cc-4294-af0e-281531c36be5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:27.961147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1920c7-68cc-4294-af0e-281531c36be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:27.984869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-fa1920c7-68cc-4294-af0e-281531c36be5 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:48:27.986056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-fa1920c7-68cc-4294-af0e-281531c36be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:28.040502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1154/2344] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 05:48:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:48:28.184543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839df148-81cd-46e2-af1d-3bfefb1f4976 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:28.186111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-839df148-81cd-46e2-af1d-3bfefb1f4976 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:28.253222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839df148-81cd-46e2-af1d-3bfefb1f4976 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:28.258107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44706bd0-ec59-4d2c-a884-bab63ed21cbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:28.258718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44706bd0-ec59-4d2c-a884-bab63ed21cbc service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:28.258999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44706bd0-ec59-4d2c-a884-bab63ed21cbc service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:28.259267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44706bd0-ec59-4d2c-a884-bab63ed21cbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:28.267606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44706bd0-ec59-4d2c-a884-bab63ed21cbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:28.269196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44706bd0-ec59-4d2c-a884-bab63ed21cbc service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:28.276926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1191/2345] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:48:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 05:48:30.788182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1920c7-68cc-4294-af0e-281531c36be5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:30.797839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1920c7-68cc-4294-af0e-281531c36be5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:30.801466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f826b24-5e5f-4c03-98a9-ef0610b0a5c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:30.802069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f826b24-5e5f-4c03-98a9-ef0610b0a5c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:30.803191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f826b24-5e5f-4c03-98a9-ef0610b0a5c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:30.803191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f826b24-5e5f-4c03-98a9-ef0610b0a5c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:30.812616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f826b24-5e5f-4c03-98a9-ef0610b0a5c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:30.814380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f826b24-5e5f-4c03-98a9-ef0610b0a5c9 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:30.823832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1155/2346] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:48:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:48:32.862045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44706bd0-ec59-4d2c-a884-bab63ed21cbc service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:32.864755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-44706bd0-ec59-4d2c-a884-bab63ed21cbc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:32.882918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44706bd0-ec59-4d2c-a884-bab63ed21cbc service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:32.891671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f31615-14b9-4459-a889-b88bf3a9caed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:32.892507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f31615-14b9-4459-a889-b88bf3a9caed service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:32.893011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f31615-14b9-4459-a889-b88bf3a9caed service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:32.893630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f31615-14b9-4459-a889-b88bf3a9caed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:32.895839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-09f31615-14b9-4459-a889-b88bf3a9caed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:32.916621 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f31615-14b9-4459-a889-b88bf3a9caed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:32.918641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f31615-14b9-4459-a889-b88bf3a9caed service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:32.930767 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1192/2347] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:48:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:48:35.708855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f826b24-5e5f-4c03-98a9-ef0610b0a5c9 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:35.722155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f826b24-5e5f-4c03-98a9-ef0610b0a5c9 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:35.727743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa4416-f026-4a04-a647-5cbdc2db3a1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:35.728339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa4416-f026-4a04-a647-5cbdc2db3a1e service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:35.728618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa4416-f026-4a04-a647-5cbdc2db3a1e service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:35.728888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa4416-f026-4a04-a647-5cbdc2db3a1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:35.730789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4aa4416-f026-4a04-a647-5cbdc2db3a1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:35.796973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa4416-f026-4a04-a647-5cbdc2db3a1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:35.798784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa4416-f026-4a04-a647-5cbdc2db3a1e service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:35.808580 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1156/2348] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:48:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:48:36.630914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f31615-14b9-4459-a889-b88bf3a9caed service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:36.641702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f31615-14b9-4459-a889-b88bf3a9caed service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:36.646921 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a8e5d-b1d7-4315-a1ca-b484fb604693 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:36.647628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8e5d-b1d7-4315-a1ca-b484fb604693 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:36.648030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8e5d-b1d7-4315-a1ca-b484fb604693 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:36.648314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8e5d-b1d7-4315-a1ca-b484fb604693 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:36.650064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d80a8e5d-b1d7-4315-a1ca-b484fb604693 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:36.662179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a8e5d-b1d7-4315-a1ca-b484fb604693 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:36.663905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a8e5d-b1d7-4315-a1ca-b484fb604693 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:36.674847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1193/2349] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:48:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:48:36.769335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa4416-f026-4a04-a647-5cbdc2db3a1e service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:36.770843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4aa4416-f026-4a04-a647-5cbdc2db3a1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:36.836049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa4416-f026-4a04-a647-5cbdc2db3a1e service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:36.840434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:36.841050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:36.841350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:36.841619 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] PATH_INFO: `/v3/users/db24ed507ded40fe828c84bc138f893b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:36.842748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] RBAC: Authorizing `identity:delete_user(user_id=db24ed507ded40fe828c84bc138f893b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:36.844639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:37.022050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:48:37.034477 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db24ed507ded40fe828c84bc138f893b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.035132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db24ed507ded40fe828c84bc138f893b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.047881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db24ed507ded40fe828c84bc138f893b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.134911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db24ed507ded40fe828c84bc138f893b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.135533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db24ed507ded40fe828c84bc138f893b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.147994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1157/2350] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:48:36 2022] DELETE /identity/v3/users/db24ed507ded40fe828c84bc138f893b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:48:37.376604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a8e5d-b1d7-4315-a1ca-b484fb604693 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:37.387810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a8e5d-b1d7-4315-a1ca-b484fb604693 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:37.445563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:37.446213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:37.446492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:37.446789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] PATH_INFO: `/v3/projects/c237fdd93f3e47dea64bd0143f09b819` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:37.447928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c237fdd93f3e47dea64bd0143f09b819)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:37.449859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:37.582901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c237fdd93f3e47dea64bd0143f09b819', 'request_id': 'req-feffead2-001f-4797-99c9-3902c40bac3b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.584673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c237fdd93f3e47dea64bd0143f09b819', 'request_id': 'req-feffead2-001f-4797-99c9-3902c40bac3b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.653371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c237fdd93f3e47dea64bd0143f09b819', 'request_id': 'req-feffead2-001f-4797-99c9-3902c40bac3b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.661047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:48:37.668265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] The token cache is being invalidate because project c237fdd93f3e47dea64bd0143f09b819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:48:37.668554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.669198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.670758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1194/2351] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:48:37 2022] DELETE /identity/v3/projects/c237fdd93f3e47dea64bd0143f09b819 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:48:37.674921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:37.676500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:37.690349 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c5ce6-ea47-46a4-8fef-3a23693fc77f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:37.746163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:37.746785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:37.747062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:37.747331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] PATH_INFO: `/v3/users/18e2a2dab2c54d7b81e6fbd0dff0765b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:37.748990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e2a2dab2c54d7b81e6fbd0dff0765b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:37.750951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:37.970515 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e2a2dab2c54d7b81e6fbd0dff0765b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.971104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e2a2dab2c54d7b81e6fbd0dff0765b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:37.983071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e2a2dab2c54d7b81e6fbd0dff0765b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:38.053251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e2a2dab2c54d7b81e6fbd0dff0765b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:38.053950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ae27e4d2-f4ec-4a1f-82b0-179d074fde8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e2a2dab2c54d7b81e6fbd0dff0765b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:38.067392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1158/2352] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:48:37 2022] DELETE /identity/v3/users/18e2a2dab2c54d7b81e6fbd0dff0765b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:48:38.342244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:48:38.343708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:48:38.359144 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffead2-001f-4797-99c9-3902c40bac3b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:48:38.416291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:48:38.416914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:48:38.417190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:48:38.417457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] PATH_INFO: `/v3/projects/cd65a0a8255d4f589832dd37a06968c5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:48:38.418581 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd65a0a8255d4f589832dd37a06968c5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:48:38.421050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:48:38.551458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd65a0a8255d4f589832dd37a06968c5', 'request_id': 'req-14adbc51-fc85-4039-9acb-1a198506bac4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:38.558036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd65a0a8255d4f589832dd37a06968c5', 'request_id': 'req-14adbc51-fc85-4039-9acb-1a198506bac4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:38.636278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd65a0a8255d4f589832dd37a06968c5', 'request_id': 'req-14adbc51-fc85-4039-9acb-1a198506bac4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:38.646863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] The token cache is being invalidate because project cd65a0a8255d4f589832dd37a06968c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:48:38.647149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:38.647771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:48:38.649340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1195/2353] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:48:38 2022] DELETE /identity/v3/projects/cd65a0a8255d4f589832dd37a06968c5 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:48:48.063603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=70,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:48:48.643744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:06.227410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa8501-7ad5-4cba-9ab3-9c35388472b9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:06.227934 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa8501-7ad5-4cba-9ab3-9c35388472b9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:06.228041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa8501-7ad5-4cba-9ab3-9c35388472b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:06.248174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3eaa8501-7ad5-4cba-9ab3-9c35388472b9 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:06.249401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3eaa8501-7ad5-4cba-9ab3-9c35388472b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:06.302767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1159/2354] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:49:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:06.315607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:06.316995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:06.384070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adbc51-fc85-4039-9acb-1a198506bac4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:06.389243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daf5ba-793d-46a3-b1e1-f6cc254644c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:06.389878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daf5ba-793d-46a3-b1e1-f6cc254644c9 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:06.390211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daf5ba-793d-46a3-b1e1-f6cc254644c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:06.390989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daf5ba-793d-46a3-b1e1-f6cc254644c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:06.399737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daf5ba-793d-46a3-b1e1-f6cc254644c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:06.401321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daf5ba-793d-46a3-b1e1-f6cc254644c9 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:06.410224 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1196/2355] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:49:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:49:07.313462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa8501-7ad5-4cba-9ab3-9c35388472b9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:07.315476 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3eaa8501-7ad5-4cba-9ab3-9c35388472b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:07.418510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa8501-7ad5-4cba-9ab3-9c35388472b9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:07.424691 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222307f0-25a7-4468-beb2-0344cc448f47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:07.425305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222307f0-25a7-4468-beb2-0344cc448f47 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:07.425577 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222307f0-25a7-4468-beb2-0344cc448f47 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:07.426575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222307f0-25a7-4468-beb2-0344cc448f47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:07.428330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-222307f0-25a7-4468-beb2-0344cc448f47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:07.450125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222307f0-25a7-4468-beb2-0344cc448f47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:07.454449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222307f0-25a7-4468-beb2-0344cc448f47 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:07.481317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1160/2356] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:16.407889 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=32 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:17.462953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=59 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:17.799214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daf5ba-793d-46a3-b1e1-f6cc254644c9 service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:17.800628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-56daf5ba-793d-46a3-b1e1-f6cc254644c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:17.816682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daf5ba-793d-46a3-b1e1-f6cc254644c9 service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:17.822160 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445c532-ddc0-4fa4-bc29-62810e6fbe44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:17.822751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445c532-ddc0-4fa4-bc29-62810e6fbe44 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:17.823024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445c532-ddc0-4fa4-bc29-62810e6fbe44 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:17.823291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445c532-ddc0-4fa4-bc29-62810e6fbe44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:17.825017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9445c532-ddc0-4fa4-bc29-62810e6fbe44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:17.882062 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445c532-ddc0-4fa4-bc29-62810e6fbe44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:17.883689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445c532-ddc0-4fa4-bc29-62810e6fbe44 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:17.893104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1197/2357] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:27.675819 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222307f0-25a7-4468-beb2-0344cc448f47 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:27.677338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-222307f0-25a7-4468-beb2-0344cc448f47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:27.738350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222307f0-25a7-4468-beb2-0344cc448f47 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:27.742097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c058771-61dd-46e0-a50d-bb8ff97d9e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:27.742730 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c058771-61dd-46e0-a50d-bb8ff97d9e1a service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:27.743005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c058771-61dd-46e0-a50d-bb8ff97d9e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:27.743279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c058771-61dd-46e0-a50d-bb8ff97d9e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:27.745646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4c058771-61dd-46e0-a50d-bb8ff97d9e1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:27.810412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c058771-61dd-46e0-a50d-bb8ff97d9e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:27.811868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c058771-61dd-46e0-a50d-bb8ff97d9e1a service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:27.822450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1161/2358] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:27.891567 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=28 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:29.714898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543545a4-34e4-4e3f-a263-59074c2fa3d0 None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:29.715991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543545a4-34e4-4e3f-a263-59074c2fa3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:29.716327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543545a4-34e4-4e3f-a263-59074c2fa3d0 None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:29.719291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1198/2359] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:49:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:49:29.725720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f5bbb-4cd8-4590-a4a4-1942fdc0a18a None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:29.726105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f5bbb-4cd8-4590-a4a4-1942fdc0a18a None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:29.726229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f5bbb-4cd8-4590-a4a4-1942fdc0a18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:29.748364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-417f5bbb-4cd8-4590-a4a4-1942fdc0a18a None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:29.749832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-417f5bbb-4cd8-4590-a4a4-1942fdc0a18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:29.769695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1162/2360] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:49:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:29.776930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543545a4-34e4-4e3f-a263-59074c2fa3d0 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:29.779080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-543545a4-34e4-4e3f-a263-59074c2fa3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:29.860557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543545a4-34e4-4e3f-a263-59074c2fa3d0 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:29.866174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b42406-5053-4324-b49b-20e40628c1b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34eb8fcef0543c282b3887dfeb74c15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:29.867254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b42406-5053-4324-b49b-20e40628c1b5 service glance] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:29.867705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b42406-5053-4324-b49b-20e40628c1b5 service glance] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:29.868134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b42406-5053-4324-b49b-20e40628c1b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:29.880544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b42406-5053-4324-b49b-20e40628c1b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:29.882646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b42406-5053-4324-b49b-20e40628c1b5 service glance] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:29.896208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1199/2361] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:30.142449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b6474-7156-49b6-ba4c-b91caf801ab9 None None] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:30.142743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b6474-7156-49b6-ba4c-b91caf801ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:30.143103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b6474-7156-49b6-ba4c-b91caf801ab9 None None] PATH_INFO: `/` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:30.149703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1163/2362] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 05:49:30.150075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4689ec2-ef74-464f-8ea8-09b4ac1e922f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:30.150429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4689ec2-ef74-464f-8ea8-09b4ac1e922f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:30.150669 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4689ec2-ef74-464f-8ea8-09b4ac1e922f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:30.172436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f4689ec2-ef74-464f-8ea8-09b4ac1e922f None None] MFA Rules not processed for user `3a0feaf83ee14071806f36d01c36b635`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:30.174374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f4689ec2-ef74-464f-8ea8-09b4ac1e922f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:30.183156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1200/2363] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:49:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:30.198072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b6474-7156-49b6-ba4c-b91caf801ab9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:30.199986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c52b6474-7156-49b6-ba4c-b91caf801ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:30.214696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b6474-7156-49b6-ba4c-b91caf801ab9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:30.218656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914b16b-7d76-4236-a9c6-dcb9623ae343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:30.219465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914b16b-7d76-4236-a9c6-dcb9623ae343 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:30.219821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914b16b-7d76-4236-a9c6-dcb9623ae343 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:30.220282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914b16b-7d76-4236-a9c6-dcb9623ae343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:30.227960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914b16b-7d76-4236-a9c6-dcb9623ae343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:30.229631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914b16b-7d76-4236-a9c6-dcb9623ae343 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:30.241140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1164/2364] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:30.347637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0fe74-d04f-4fdf-9313-a38360c5c14d None None] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:30.348124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0fe74-d04f-4fdf-9313-a38360c5c14d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:30.348470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0fe74-d04f-4fdf-9313-a38360c5c14d None None] PATH_INFO: `/` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:30.350250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1201/2365] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 05:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 05:49:30.358862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e8283-d60d-42bb-82e2-a99fb5cb6584 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:30.359258 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e8283-d60d-42bb-82e2-a99fb5cb6584 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:30.359607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e8283-d60d-42bb-82e2-a99fb5cb6584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:30.380313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-579e8283-d60d-42bb-82e2-a99fb5cb6584 None None] MFA Rules not processed for user `fba4f0473f4c4d3bb2d50b09f3de9837`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:30.382387 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-579e8283-d60d-42bb-82e2-a99fb5cb6584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:30.392516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1165/2366] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 05:49:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:30.403031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0fe74-d04f-4fdf-9313-a38360c5c14d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:30.414173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0fe74-d04f-4fdf-9313-a38360c5c14d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:30.417844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f0d7b-fa62-4269-94e4-cb9a6b711f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:30.418508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f0d7b-fa62-4269-94e4-cb9a6b711f0f service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:30.418780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f0d7b-fa62-4269-94e4-cb9a6b711f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:30.419049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f0d7b-fa62-4269-94e4-cb9a6b711f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:30.429977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f0d7b-fa62-4269-94e4-cb9a6b711f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:30.431478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f0d7b-fa62-4269-94e4-cb9a6b711f0f service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:30.443575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1202/2367] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:32.513100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e8283-d60d-42bb-82e2-a99fb5cb6584 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:32.514845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-579e8283-d60d-42bb-82e2-a99fb5cb6584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:32.531353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e8283-d60d-42bb-82e2-a99fb5cb6584 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:32.536097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:32.536813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:32.537308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:32.537646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] PATH_INFO: `/v3/users/c1b48a973c20470a9af23c77e3da5b72` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:32.538837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1b48a973c20470a9af23c77e3da5b72)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:32.540817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:32.754846 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1b48a973c20470a9af23c77e3da5b72'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:32.755532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1b48a973c20470a9af23c77e3da5b72'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:32.776101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1b48a973c20470a9af23c77e3da5b72'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:32.913253 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1b48a973c20470a9af23c77e3da5b72'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:32.913911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-003cbade-dd69-46db-b2a5-4d6d1882aca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1b48a973c20470a9af23c77e3da5b72'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:32.920723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:32.929153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1166/2368] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:49:32 2022] DELETE /identity/v3/users/c1b48a973c20470a9af23c77e3da5b72 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:33.187825 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f0d7b-fa62-4269-94e4-cb9a6b711f0f service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:33.200080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f0d7b-fa62-4269-94e4-cb9a6b711f0f service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:33.257094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:33.257719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:33.257996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:33.258301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] PATH_INFO: `/v3/projects/689c4759cb5943b79f2637a8e29e230e` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:33.259445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=689c4759cb5943b79f2637a8e29e230e)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:33.261358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:33.361201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '689c4759cb5943b79f2637a8e29e230e', 'request_id': 'req-e61d347e-8acb-44b9-831a-25ca9de8b414'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:33.368361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '689c4759cb5943b79f2637a8e29e230e', 'request_id': 'req-e61d347e-8acb-44b9-831a-25ca9de8b414'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:33.432129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '689c4759cb5943b79f2637a8e29e230e', 'request_id': 'req-e61d347e-8acb-44b9-831a-25ca9de8b414'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:33.448907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] The token cache is being invalidate because project 689c4759cb5943b79f2637a8e29e230e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:49:33.449383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:33.450073 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e61d347e-8acb-44b9-831a-25ca9de8b414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:33.451935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1203/2369] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:49:33 2022] DELETE /identity/v3/projects/689c4759cb5943b79f2637a8e29e230e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:33.457303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fafeb0-b7f0-4206-9e92-82b49599253d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:33.457864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fafeb0-b7f0-4206-9e92-82b49599253d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:33.458400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fafeb0-b7f0-4206-9e92-82b49599253d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:33.483156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-58fafeb0-b7f0-4206-9e92-82b49599253d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:33.484460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-58fafeb0-b7f0-4206-9e92-82b49599253d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:33.558514 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1167/2370] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:33.567591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e63e7-fcf0-4814-84c4-536a1ec83675 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:33.567881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e63e7-fcf0-4814-84c4-536a1ec83675 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:33.568151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e63e7-fcf0-4814-84c4-536a1ec83675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:33.588459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-017e63e7-fcf0-4814-84c4-536a1ec83675 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:33.589663 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-017e63e7-fcf0-4814-84c4-536a1ec83675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:33.598580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1204/2371] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:33.602864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fafeb0-b7f0-4206-9e92-82b49599253d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:33.612582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fafeb0-b7f0-4206-9e92-82b49599253d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:33.617120 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b66fc-533d-48ff-8b68-e8fe2bcc385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:33.617734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b66fc-533d-48ff-8b68-e8fe2bcc385b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:33.618014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b66fc-533d-48ff-8b68-e8fe2bcc385b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:33.618326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b66fc-533d-48ff-8b68-e8fe2bcc385b admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:33.618964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b66fc-533d-48ff-8b68-e8fe2bcc385b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:49:33.619269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b66fc-533d-48ff-8b68-e8fe2bcc385b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:33.620852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b66fc-533d-48ff-8b68-e8fe2bcc385b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:33.632142 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1168/2372] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:49:33.636478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e63e7-fcf0-4814-84c4-536a1ec83675 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:33.646213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e63e7-fcf0-4814-84c4-536a1ec83675 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:33.650851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b7c28-c665-4315-98e3-d1cb3621e625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:33.651434 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b7c28-c665-4315-98e3-d1cb3621e625 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:33.651707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b7c28-c665-4315-98e3-d1cb3621e625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:33.651976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b7c28-c665-4315-98e3-d1cb3621e625 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:33.652607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b7c28-c665-4315-98e3-d1cb3621e625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:33.654539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b7c28-c665-4315-98e3-d1cb3621e625 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:33.659516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:33.729900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1205/2373] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:49:33 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:33.734028 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b66fc-533d-48ff-8b68-e8fe2bcc385b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:33.744031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b66fc-533d-48ff-8b68-e8fe2bcc385b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:33.800249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c32243-92c6-4cbc-88b9-e810f9df40cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:33.800836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c32243-92c6-4cbc-88b9-e810f9df40cf admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:33.801118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c32243-92c6-4cbc-88b9-e810f9df40cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:33.801390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c32243-92c6-4cbc-88b9-e810f9df40cf admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:33.802020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c32243-92c6-4cbc-88b9-e810f9df40cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:33.803975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c32243-92c6-4cbc-88b9-e810f9df40cf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:33.897269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1169/2374] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:49:33 2022] POST /identity/v3/users => generated 283 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:33.900821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b7c28-c665-4315-98e3-d1cb3621e625 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:33.911578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b7c28-c665-4315-98e3-d1cb3621e625 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:33.916332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f1c61-86b6-4fd3-bf0d-68196e68510d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:33.916933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f1c61-86b6-4fd3-bf0d-68196e68510d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:33.917213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f1c61-86b6-4fd3-bf0d-68196e68510d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:33.917499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f1c61-86b6-4fd3-bf0d-68196e68510d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:33.918201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f1c61-86b6-4fd3-bf0d-68196e68510d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:33.919808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f1c61-86b6-4fd3-bf0d-68196e68510d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:33.930105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1206/2375] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:33.934009 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c32243-92c6-4cbc-88b9-e810f9df40cf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:33.944876 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c32243-92c6-4cbc-88b9-e810f9df40cf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:33.949532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855d6d0-ed99-47a1-8582-8c43f076d89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:33.950132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855d6d0-ed99-47a1-8582-8c43f076d89e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:33.950441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855d6d0-ed99-47a1-8582-8c43f076d89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:33.950705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855d6d0-ed99-47a1-8582-8c43f076d89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:33.951346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8855d6d0-ed99-47a1-8582-8c43f076d89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:33.953461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8855d6d0-ed99-47a1-8582-8c43f076d89e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:33.964266 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1170/2376] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:49:33.968543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f1c61-86b6-4fd3-bf0d-68196e68510d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:33.978782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f1c61-86b6-4fd3-bf0d-68196e68510d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:33.983391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632585d-7550-4f6d-ab42-121021941354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:33.984030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632585d-7550-4f6d-ab42-121021941354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:33.984343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632585d-7550-4f6d-ab42-121021941354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:33.984602 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632585d-7550-4f6d-ab42-121021941354 admin admin] PATH_INFO: `/v3/projects/2754c2643d2540729ee56b69cb097789/users/70ffc60e492840e88c7e1c733aef5b92/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:33.999257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632585d-7550-4f6d-ab42-121021941354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2754c2643d2540729ee56b69cb097789, user_id=70ffc60e492840e88c7e1c733aef5b92, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:34.002511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632585d-7550-4f6d-ab42-121021941354 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:34.076664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1207/2377] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:33 2022] PUT /identity/v3/projects/2754c2643d2540729ee56b69cb097789/users/70ffc60e492840e88c7e1c733aef5b92/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:34.087631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261a30d-e5d5-4e69-9696-a661f48637f3 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:34.087943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261a30d-e5d5-4e69-9696-a661f48637f3 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:34.088216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261a30d-e5d5-4e69-9696-a661f48637f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:34.116123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-c261a30d-e5d5-4e69-9696-a661f48637f3 None None] MFA Rules not processed for user `70ffc60e492840e88c7e1c733aef5b92`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:34.117338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c261a30d-e5d5-4e69-9696-a661f48637f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:34.164070 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:34.180608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1171/2378] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:34.185181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632585d-7550-4f6d-ab42-121021941354 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:34.246071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632585d-7550-4f6d-ab42-121021941354 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:34.251381 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689a87d-cb2d-4a44-bb19-705482ed0b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:34.251978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689a87d-cb2d-4a44-bb19-705482ed0b0c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:34.252256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689a87d-cb2d-4a44-bb19-705482ed0b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:34.252528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689a87d-cb2d-4a44-bb19-705482ed0b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:34.253162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689a87d-cb2d-4a44-bb19-705482ed0b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:34.255093 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689a87d-cb2d-4a44-bb19-705482ed0b0c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:34.329801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1208/2379] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:49:34 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:34.333935 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261a30d-e5d5-4e69-9696-a661f48637f3 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:34.344870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261a30d-e5d5-4e69-9696-a661f48637f3 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:34.402356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209bd63e-4261-4935-ac05-1d8966617566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:34.403550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bd63e-4261-4935-ac05-1d8966617566 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:34.406011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bd63e-4261-4935-ac05-1d8966617566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:34.406011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bd63e-4261-4935-ac05-1d8966617566 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:34.406011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209bd63e-4261-4935-ac05-1d8966617566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:34.407030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209bd63e-4261-4935-ac05-1d8966617566 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:34.498531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1172/2380] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:49:34 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:34.500354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689a87d-cb2d-4a44-bb19-705482ed0b0c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:34.511657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689a87d-cb2d-4a44-bb19-705482ed0b0c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:34.517541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b8cc1-d3c9-4c0f-8b72-c05ab275e6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:34.518751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b8cc1-d3c9-4c0f-8b72-c05ab275e6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:34.519182 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b8cc1-d3c9-4c0f-8b72-c05ab275e6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:34.519598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b8cc1-d3c9-4c0f-8b72-c05ab275e6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:34.520394 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b8cc1-d3c9-4c0f-8b72-c05ab275e6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:34.522350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b8cc1-d3c9-4c0f-8b72-c05ab275e6b7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:34.535404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1209/2381] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:49:34.539187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209bd63e-4261-4935-ac05-1d8966617566 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:34.553011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209bd63e-4261-4935-ac05-1d8966617566 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:34.559460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c6aba-9f5e-40d2-aa4a-61856cfd9467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:34.560140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c6aba-9f5e-40d2-aa4a-61856cfd9467 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:34.560429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c6aba-9f5e-40d2-aa4a-61856cfd9467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:34.560706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c6aba-9f5e-40d2-aa4a-61856cfd9467 admin admin] PATH_INFO: `/v3/projects/a562fdd7f9224e4fa2268646bbdc51a2/users/b1a5fa95fa424149bcefc3a0b94c30a7/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:34.584513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c6aba-9f5e-40d2-aa4a-61856cfd9467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a562fdd7f9224e4fa2268646bbdc51a2, user_id=b1a5fa95fa424149bcefc3a0b94c30a7, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:34.588443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c6aba-9f5e-40d2-aa4a-61856cfd9467 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:34.680692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1173/2382] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:34 2022] PUT /identity/v3/projects/a562fdd7f9224e4fa2268646bbdc51a2/users/b1a5fa95fa424149bcefc3a0b94c30a7/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:34.685332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b8cc1-d3c9-4c0f-8b72-c05ab275e6b7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:34.697352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b8cc1-d3c9-4c0f-8b72-c05ab275e6b7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:34.772971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1ee2-57e8-4e4d-a332-7052d2bfa7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:34.772971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1ee2-57e8-4e4d-a332-7052d2bfa7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:34.772971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1ee2-57e8-4e4d-a332-7052d2bfa7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:34.772971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1ee2-57e8-4e4d-a332-7052d2bfa7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:34.773469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e1ee2-57e8-4e4d-a332-7052d2bfa7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:34.773469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e1ee2-57e8-4e4d-a332-7052d2bfa7e9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:34.781588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1210/2383] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:34.783453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c6aba-9f5e-40d2-aa4a-61856cfd9467 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:34.794460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c6aba-9f5e-40d2-aa4a-61856cfd9467 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:34.800179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281968a-4e02-42ea-8ab9-83cb67c27858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:34.801981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281968a-4e02-42ea-8ab9-83cb67c27858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:34.801981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281968a-4e02-42ea-8ab9-83cb67c27858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:34.801981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281968a-4e02-42ea-8ab9-83cb67c27858 admin admin] PATH_INFO: `/v3/projects/a562fdd7f9224e4fa2268646bbdc51a2/users/b1a5fa95fa424149bcefc3a0b94c30a7/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:34.803785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281968a-4e02-42ea-8ab9-83cb67c27858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a562fdd7f9224e4fa2268646bbdc51a2, user_id=b1a5fa95fa424149bcefc3a0b94c30a7, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:34.806428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281968a-4e02-42ea-8ab9-83cb67c27858 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:34.818478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1174/2384] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:34 2022] PUT /identity/v3/projects/a562fdd7f9224e4fa2268646bbdc51a2/users/b1a5fa95fa424149bcefc3a0b94c30a7/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:34.828891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff224640-69ba-465e-a5b6-dddc2f92e582 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:34.829174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff224640-69ba-465e-a5b6-dddc2f92e582 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:34.829450 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff224640-69ba-465e-a5b6-dddc2f92e582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:34.859743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ff224640-69ba-465e-a5b6-dddc2f92e582 None None] MFA Rules not processed for user `b1a5fa95fa424149bcefc3a0b94c30a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:34.861004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ff224640-69ba-465e-a5b6-dddc2f92e582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:34.869199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:34.945661 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1211/2385] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:34.957328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281968a-4e02-42ea-8ab9-83cb67c27858 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:34.959176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0281968a-4e02-42ea-8ab9-83cb67c27858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:35.038354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281968a-4e02-42ea-8ab9-83cb67c27858 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:35.045227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1848bf4-575f-423c-a3ab-526f4f44ee58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:35.045941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1848bf4-575f-423c-a3ab-526f4f44ee58 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:35.046427 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1848bf4-575f-423c-a3ab-526f4f44ee58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:35.046782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1848bf4-575f-423c-a3ab-526f4f44ee58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:35.057773 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1848bf4-575f-423c-a3ab-526f4f44ee58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:35.064834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1848bf4-575f-423c-a3ab-526f4f44ee58 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:35.072159 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1175/2386] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:35.519100 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff224640-69ba-465e-a5b6-dddc2f92e582 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:35.530187 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff224640-69ba-465e-a5b6-dddc2f92e582 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:35.534275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e5af0-b227-4539-8899-221d50e63c8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:35.539694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e5af0-b227-4539-8899-221d50e63c8a service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:35.539694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e5af0-b227-4539-8899-221d50e63c8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:35.539694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e5af0-b227-4539-8899-221d50e63c8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:35.607316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e5af0-b227-4539-8899-221d50e63c8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:35.608984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e5af0-b227-4539-8899-221d50e63c8a service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:35.622170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1212/2387] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:36.188360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf75c7-7007-4585-a430-11ccf4785b56 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:36.188894 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf75c7-7007-4585-a430-11ccf4785b56 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:36.188997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf75c7-7007-4585-a430-11ccf4785b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:36.209963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-97bf75c7-7007-4585-a430-11ccf4785b56 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:36.211290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-97bf75c7-7007-4585-a430-11ccf4785b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:36.277820 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:36.295418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1176/2388] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:49:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:36.330647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e5af0-b227-4539-8899-221d50e63c8a service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:36.332899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-df8e5af0-b227-4539-8899-221d50e63c8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:36.559233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e5af0-b227-4539-8899-221d50e63c8a service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:36.565169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ba941-2ab5-401d-9a7f-b6cf6eecdcfb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:36.570866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ba941-2ab5-401d-9a7f-b6cf6eecdcfb service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:36.570866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ba941-2ab5-401d-9a7f-b6cf6eecdcfb service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:36.570866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ba941-2ab5-401d-9a7f-b6cf6eecdcfb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:36.579039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ba941-2ab5-401d-9a7f-b6cf6eecdcfb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:36.580932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ba941-2ab5-401d-9a7f-b6cf6eecdcfb service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:36.590913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1213/2389] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:49:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 05:49:44.970672 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf75c7-7007-4585-a430-11ccf4785b56 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:44.972394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-97bf75c7-7007-4585-a430-11ccf4785b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:45.048922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf75c7-7007-4585-a430-11ccf4785b56 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:45.053051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b7caf-a148-4e2d-a867-c9a5f6ba2c97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:45.053822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b7caf-a148-4e2d-a867-c9a5f6ba2c97 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:45.054201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b7caf-a148-4e2d-a867-c9a5f6ba2c97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:45.054517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b7caf-a148-4e2d-a867-c9a5f6ba2c97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:45.056734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4a0b7caf-a148-4e2d-a867-c9a5f6ba2c97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:45.159377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b7caf-a148-4e2d-a867-c9a5f6ba2c97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:45.161379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b7caf-a148-4e2d-a867-c9a5f6ba2c97 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:45.185976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1177/2390] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:46.587003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=85 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:48.623335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ba941-2ab5-401d-9a7f-b6cf6eecdcfb service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:48.636913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ba941-2ab5-401d-9a7f-b6cf6eecdcfb service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:48.642519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:48.643165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:48.643446 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:48.643718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] PATH_INFO: `/v3/users/70ffc60e492840e88c7e1c733aef5b92` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:48.645082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70ffc60e492840e88c7e1c733aef5b92)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:48.647031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:48.869287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70ffc60e492840e88c7e1c733aef5b92'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:48.887496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70ffc60e492840e88c7e1c733aef5b92'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:48.890978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70ffc60e492840e88c7e1c733aef5b92'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:48.945338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70ffc60e492840e88c7e1c733aef5b92'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:48.946172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70ffc60e492840e88c7e1c733aef5b92'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:48.960499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1214/2391] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:49:48 2022] DELETE /identity/v3/users/70ffc60e492840e88c7e1c733aef5b92 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:48.965132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56f5b1-c74b-4dc9-b3eb-a9fb016e0595 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:48.965747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56f5b1-c74b-4dc9-b3eb-a9fb016e0595 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:48.965747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56f5b1-c74b-4dc9-b3eb-a9fb016e0595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:48.986913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0f56f5b1-c74b-4dc9-b3eb-a9fb016e0595 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:48.988279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0f56f5b1-c74b-4dc9-b3eb-a9fb016e0595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:49.057373 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1178/2392] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:49.067162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:49.068560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:49.108676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56f5b1-c74b-4dc9-b3eb-a9fb016e0595 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:49.108676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0f56f5b1-c74b-4dc9-b3eb-a9fb016e0595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:49.118069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56f5b1-c74b-4dc9-b3eb-a9fb016e0595 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:49.123550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:49.124225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:49.124551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:49.124830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] PATH_INFO: `/v3/users/c66c7ddbf18d4f1fb8c890f548031f40` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:49.125995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66c7ddbf18d4f1fb8c890f548031f40)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:49.128048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:49.136928 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee93be-61f3-49c6-b717-13245a8f1e32 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:49.140644 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad263bf-6737-436d-8baf-8e021240ad6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:49.141242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad263bf-6737-436d-8baf-8e021240ad6c service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:49.141515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad263bf-6737-436d-8baf-8e021240ad6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:49.141781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad263bf-6737-436d-8baf-8e021240ad6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:49.151262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad263bf-6737-436d-8baf-8e021240ad6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:49.152786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad263bf-6737-436d-8baf-8e021240ad6c service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:49.162811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1215/2393] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:49.324176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:49.333537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66c7ddbf18d4f1fb8c890f548031f40'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:49.334247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66c7ddbf18d4f1fb8c890f548031f40'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:49.347286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66c7ddbf18d4f1fb8c890f548031f40'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:49.443999 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66c7ddbf18d4f1fb8c890f548031f40'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:49.444912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66c7ddbf18d4f1fb8c890f548031f40'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:49.465454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1179/2394] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:49:49 2022] DELETE /identity/v3/users/c66c7ddbf18d4f1fb8c890f548031f40 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:49.666249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad263bf-6737-436d-8baf-8e021240ad6c service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:49.666249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8ad263bf-6737-436d-8baf-8e021240ad6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:49.689092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad263bf-6737-436d-8baf-8e021240ad6c service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:49.750012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:49.750690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:49.750961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:49.751230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] PATH_INFO: `/v3/projects/2754c2643d2540729ee56b69cb097789` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:49.752499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2754c2643d2540729ee56b69cb097789)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:49.754441 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:49.771222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:49.810706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:49.812747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:49.830061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fc7a3-5baa-419a-8050-38d1ff03e0c4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:49.837420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:49.837420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:49.837420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:49.837420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] PATH_INFO: `/v3/projects/f7b850d4b5df4a4e93720cb855add44f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:49.840243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b850d4b5df4a4e93720cb855add44f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:49.842379 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:49.909252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2754c2643d2540729ee56b69cb097789', 'request_id': 'req-df029b47-cb99-4941-bad1-6ff8aedaebb9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:49.915604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2754c2643d2540729ee56b69cb097789', 'request_id': 'req-df029b47-cb99-4941-bad1-6ff8aedaebb9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:49.981514 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b850d4b5df4a4e93720cb855add44f', 'request_id': 'req-0e53c3cf-675b-475f-9e53-87a3ed070560'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:49.988700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b850d4b5df4a4e93720cb855add44f', 'request_id': 'req-0e53c3cf-675b-475f-9e53-87a3ed070560'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.000366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2754c2643d2540729ee56b69cb097789', 'request_id': 'req-df029b47-cb99-4941-bad1-6ff8aedaebb9'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.011589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] The token cache is being invalidate because project 2754c2643d2540729ee56b69cb097789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:49:50.011877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.012505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.014043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1216/2395] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:49:49 2022] DELETE /identity/v3/projects/2754c2643d2540729ee56b69cb097789 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:50.017838 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:50.019246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:50.077300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b850d4b5df4a4e93720cb855add44f', 'request_id': 'req-0e53c3cf-675b-475f-9e53-87a3ed070560'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.088602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] The token cache is being invalidate because project f7b850d4b5df4a4e93720cb855add44f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:49:50.088891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.089533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0e53c3cf-675b-475f-9e53-87a3ed070560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.091055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1180/2396] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:49:49 2022] DELETE /identity/v3/projects/f7b850d4b5df4a4e93720cb855add44f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:50.091574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df029b47-cb99-4941-bad1-6ff8aedaebb9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:50.096899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d28d6-dcdf-4bfa-8caa-03bd56edb947 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:50.097176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d28d6-dcdf-4bfa-8caa-03bd56edb947 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:50.097444 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d28d6-dcdf-4bfa-8caa-03bd56edb947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:50.117538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-686d28d6-dcdf-4bfa-8caa-03bd56edb947 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:50.118762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-686d28d6-dcdf-4bfa-8caa-03bd56edb947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:50.155468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:50.156253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:50.156533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:50.156804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] PATH_INFO: `/v3/users/b1a5fa95fa424149bcefc3a0b94c30a7` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:50.157992 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a5fa95fa424149bcefc3a0b94c30a7)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:50.160163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:50.179155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1181/2397] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:50.187689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c739f9b-8c43-4b36-b990-66ee15e6cd9d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:50.187977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c739f9b-8c43-4b36-b990-66ee15e6cd9d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:50.188251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c739f9b-8c43-4b36-b990-66ee15e6cd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:50.207717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3c739f9b-8c43-4b36-b990-66ee15e6cd9d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:50.208926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3c739f9b-8c43-4b36-b990-66ee15e6cd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:50.216917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1182/2398] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:50.221038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c739f9b-8c43-4b36-b990-66ee15e6cd9d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:50.229975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c739f9b-8c43-4b36-b990-66ee15e6cd9d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:50.235498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022017c-9bdd-4d59-bbd7-f5f3aecb9d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:50.236100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022017c-9bdd-4d59-bbd7-f5f3aecb9d38 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:50.236383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022017c-9bdd-4d59-bbd7-f5f3aecb9d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:50.236659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022017c-9bdd-4d59-bbd7-f5f3aecb9d38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:50.237297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022017c-9bdd-4d59-bbd7-f5f3aecb9d38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:49:50.237604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022017c-9bdd-4d59-bbd7-f5f3aecb9d38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:50.239252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022017c-9bdd-4d59-bbd7-f5f3aecb9d38 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:50.257419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1183/2399] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:49:50.262324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022017c-9bdd-4d59-bbd7-f5f3aecb9d38 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:50.274834 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022017c-9bdd-4d59-bbd7-f5f3aecb9d38 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:50.280453 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745389e-68cd-4a7e-bedc-c3e9de00ed80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:50.281130 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745389e-68cd-4a7e-bedc-c3e9de00ed80 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:50.281413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745389e-68cd-4a7e-bedc-c3e9de00ed80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:50.281689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745389e-68cd-4a7e-bedc-c3e9de00ed80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:50.282383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745389e-68cd-4a7e-bedc-c3e9de00ed80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:50.284405 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745389e-68cd-4a7e-bedc-c3e9de00ed80 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:50.301014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:50.354539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a5fa95fa424149bcefc3a0b94c30a7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.360402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:50.367442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a5fa95fa424149bcefc3a0b94c30a7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.368200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a5fa95fa424149bcefc3a0b94c30a7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.374437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1184/2400] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:49:50 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:49:50.375515 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745389e-68cd-4a7e-bedc-c3e9de00ed80 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:50.377139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b745389e-68cd-4a7e-bedc-c3e9de00ed80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:50.439454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745389e-68cd-4a7e-bedc-c3e9de00ed80 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:50.444449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608746b8-19c0-454d-b990-fb3bd1fc9b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:50.445051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608746b8-19c0-454d-b990-fb3bd1fc9b7a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:50.445333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608746b8-19c0-454d-b990-fb3bd1fc9b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:50.445607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608746b8-19c0-454d-b990-fb3bd1fc9b7a admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:50.446263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608746b8-19c0-454d-b990-fb3bd1fc9b7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:50.448252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608746b8-19c0-454d-b990-fb3bd1fc9b7a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:50.454646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a5fa95fa424149bcefc3a0b94c30a7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.455440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a5fa95fa424149bcefc3a0b94c30a7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:50.468790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1217/2401] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:49:50 2022] DELETE /identity/v3/users/b1a5fa95fa424149bcefc3a0b94c30a7 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:50.541010 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1185/2402] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:49:50 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:50.545010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:50.605769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b40c27-35eb-4916-9cb1-fdf3beca6d1d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:50.613342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e7ce6-1754-4adc-958c-bd4ce7b4376a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:50.613963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e7ce6-1754-4adc-958c-bd4ce7b4376a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:50.614359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e7ce6-1754-4adc-958c-bd4ce7b4376a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:50.614631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e7ce6-1754-4adc-958c-bd4ce7b4376a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:50.615319 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e7ce6-1754-4adc-958c-bd4ce7b4376a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:50.616967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e7ce6-1754-4adc-958c-bd4ce7b4376a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:50.628959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1218/2403] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:50.633122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608746b8-19c0-454d-b990-fb3bd1fc9b7a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:50.643357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608746b8-19c0-454d-b990-fb3bd1fc9b7a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:50.648069 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d8f78-bf4a-4f6a-bd80-95addca8959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:50.943604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d8f78-bf4a-4f6a-bd80-95addca8959b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:50.943604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d8f78-bf4a-4f6a-bd80-95addca8959b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:50.943604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d8f78-bf4a-4f6a-bd80-95addca8959b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:50.943604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d8f78-bf4a-4f6a-bd80-95addca8959b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:50.943604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d8f78-bf4a-4f6a-bd80-95addca8959b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:50.943604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1186/2404] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:50.947096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e7ce6-1754-4adc-958c-bd4ce7b4376a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:50.960994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e7ce6-1754-4adc-958c-bd4ce7b4376a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:50.967890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bbba8-4134-4af5-b2d8-8cbe669a04e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:50.969193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bbba8-4134-4af5-b2d8-8cbe669a04e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:50.969490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bbba8-4134-4af5-b2d8-8cbe669a04e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:50.969762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bbba8-4134-4af5-b2d8-8cbe669a04e6 admin admin] PATH_INFO: `/v3/projects/b0b81dfe4d85414ea48e4e69b7a14a5e/users/e8e3d3ed9b564e6c92835538cb9693a0/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:50.986241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6bbba8-4134-4af5-b2d8-8cbe669a04e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b81dfe4d85414ea48e4e69b7a14a5e, user_id=e8e3d3ed9b564e6c92835538cb9693a0, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:50.989395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6bbba8-4134-4af5-b2d8-8cbe669a04e6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:51.063699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496db77-ebe4-4ed7-b23e-484792b22595 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:51.064161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496db77-ebe4-4ed7-b23e-484792b22595 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:51.064255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496db77-ebe4-4ed7-b23e-484792b22595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:51.085490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4496db77-ebe4-4ed7-b23e-484792b22595 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:51.086748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4496db77-ebe4-4ed7-b23e-484792b22595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:51.145832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1187/2405] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 05:49:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:51.158403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496db77-ebe4-4ed7-b23e-484792b22595 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:51.159758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4496db77-ebe4-4ed7-b23e-484792b22595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:51.229240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496db77-ebe4-4ed7-b23e-484792b22595 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:51.234263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f18c67-73d7-4480-b4b7-aba10922e1f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:51.234864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f18c67-73d7-4480-b4b7-aba10922e1f6 service swift] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:51.235145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f18c67-73d7-4480-b4b7-aba10922e1f6 service swift] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:51.235409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f18c67-73d7-4480-b4b7-aba10922e1f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:51.243867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f18c67-73d7-4480-b4b7-aba10922e1f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:51.245468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f18c67-73d7-4480-b4b7-aba10922e1f6 service swift] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:51.253959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1188/2406] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:49:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 05:49:51.517773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1219/2407] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:50 2022] PUT /identity/v3/projects/b0b81dfe4d85414ea48e4e69b7a14a5e/users/e8e3d3ed9b564e6c92835538cb9693a0/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:51.528243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf27bb-d59e-40ed-8298-12c255b07a0e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:51.528642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf27bb-d59e-40ed-8298-12c255b07a0e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:51.529024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf27bb-d59e-40ed-8298-12c255b07a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:51.558130 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:51.566906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ccdf27bb-d59e-40ed-8298-12c255b07a0e None None] MFA Rules not processed for user `e8e3d3ed9b564e6c92835538cb9693a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:51.568349 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ccdf27bb-d59e-40ed-8298-12c255b07a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:51.634417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1189/2408] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:51.638868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bbba8-4134-4af5-b2d8-8cbe669a04e6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:51.701245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bbba8-4134-4af5-b2d8-8cbe669a04e6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:51.706091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e52090-d026-493f-9397-c046b8ca4328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:51.706719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e52090-d026-493f-9397-c046b8ca4328 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:51.710115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e52090-d026-493f-9397-c046b8ca4328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:51.710115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e52090-d026-493f-9397-c046b8ca4328 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:51.710115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e52090-d026-493f-9397-c046b8ca4328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:51.710115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e52090-d026-493f-9397-c046b8ca4328 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:51.781981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf27bb-d59e-40ed-8298-12c255b07a0e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:51.783423 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ccdf27bb-d59e-40ed-8298-12c255b07a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:51.802613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf27bb-d59e-40ed-8298-12c255b07a0e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:51.806733 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:51.807456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:51.807705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:51.807907 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] PATH_INFO: `/v3/projects/a562fdd7f9224e4fa2268646bbdc51a2` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:51.809123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a562fdd7f9224e4fa2268646bbdc51a2)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:51.811166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:51.828117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1220/2409] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:49:51 2022] POST /identity/v3/projects => generated 367 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:51.832515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e52090-d026-493f-9397-c046b8ca4328 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:51.863597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:51.895859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e52090-d026-493f-9397-c046b8ca4328 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:51.900559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a3eb5-764a-4396-9b68-aabbc6da2b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:51.901164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a3eb5-764a-4396-9b68-aabbc6da2b18 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:51.901438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a3eb5-764a-4396-9b68-aabbc6da2b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:51.901709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a3eb5-764a-4396-9b68-aabbc6da2b18 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:51.902383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a3eb5-764a-4396-9b68-aabbc6da2b18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:51.904417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a3eb5-764a-4396-9b68-aabbc6da2b18 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:51.947229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a562fdd7f9224e4fa2268646bbdc51a2', 'request_id': 'req-f5bc0e53-5fe3-4906-84be-128a4dd74594'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:51.954707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a562fdd7f9224e4fa2268646bbdc51a2', 'request_id': 'req-f5bc0e53-5fe3-4906-84be-128a4dd74594'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:52.066527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a562fdd7f9224e4fa2268646bbdc51a2', 'request_id': 'req-f5bc0e53-5fe3-4906-84be-128a4dd74594'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:52.075132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1221/2410] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:49:51 2022] POST /identity/v3/users => generated 290 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:52.083605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] The token cache is being invalidate because project a562fdd7f9224e4fa2268646bbdc51a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:49:52.083966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:52.084714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f5bc0e53-5fe3-4906-84be-128a4dd74594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:52.086571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1190/2411] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:49:51 2022] DELETE /identity/v3/projects/a562fdd7f9224e4fa2268646bbdc51a2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:52.089749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a3eb5-764a-4396-9b68-aabbc6da2b18 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.090577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7c1a3eb5-764a-4396-9b68-aabbc6da2b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:52.091340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e10a3-393a-4b41-9423-3cef22bf0207 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.091783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e10a3-393a-4b41-9423-3cef22bf0207 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.092313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e10a3-393a-4b41-9423-3cef22bf0207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.105180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a3eb5-764a-4396-9b68-aabbc6da2b18 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.114405 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0c1e10a3-393a-4b41-9423-3cef22bf0207 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:52.115775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0c1e10a3-393a-4b41-9423-3cef22bf0207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:52.163736 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd9dde-e06a-4a63-8195-651b82537d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.164585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd9dde-e06a-4a63-8195-651b82537d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.165063 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd9dde-e06a-4a63-8195-651b82537d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.165336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd9dde-e06a-4a63-8195-651b82537d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.166028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd9dde-e06a-4a63-8195-651b82537d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.167999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd9dde-e06a-4a63-8195-651b82537d4c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.180864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1222/2412] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:49:52.184856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd9dde-e06a-4a63-8195-651b82537d4c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.195428 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd9dde-e06a-4a63-8195-651b82537d4c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.200485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4c4b3-1391-4641-a1ef-98492fbe6a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.201124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4c4b3-1391-4641-a1ef-98492fbe6a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.201403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4c4b3-1391-4641-a1ef-98492fbe6a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.203104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4c4b3-1391-4641-a1ef-98492fbe6a8a admin admin] PATH_INFO: `/v3/projects/26e91c6957334581beb2200ba6f4ffc3/users/60401da2abf34807a3d7bb8cf4b82856/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.214280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1191/2413] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:52.219398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b770e7-0c7c-41b7-af16-973ed9a69df5 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.219682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b770e7-0c7c-41b7-af16-973ed9a69df5 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.219952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b770e7-0c7c-41b7-af16-973ed9a69df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.221673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4c4b3-1391-4641-a1ef-98492fbe6a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e91c6957334581beb2200ba6f4ffc3, user_id=60401da2abf34807a3d7bb8cf4b82856, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.224582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4c4b3-1391-4641-a1ef-98492fbe6a8a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.240667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f8b770e7-0c7c-41b7-af16-973ed9a69df5 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:52.241860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f8b770e7-0c7c-41b7-af16-973ed9a69df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:52.250254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1192/2414] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:52.254242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b770e7-0c7c-41b7-af16-973ed9a69df5 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.263102 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b770e7-0c7c-41b7-af16-973ed9a69df5 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.267807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8562a-b89e-41f5-8855-3da9d077b04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.268389 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8562a-b89e-41f5-8855-3da9d077b04b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.275883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8562a-b89e-41f5-8855-3da9d077b04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.275883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8562a-b89e-41f5-8855-3da9d077b04b admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.275883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8562a-b89e-41f5-8855-3da9d077b04b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:49:52.275883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8562a-b89e-41f5-8855-3da9d077b04b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.275883 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8562a-b89e-41f5-8855-3da9d077b04b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.282603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1193/2415] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:49:52.286463 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8562a-b89e-41f5-8855-3da9d077b04b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.291084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:52.296661 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8562a-b89e-41f5-8855-3da9d077b04b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.301627 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6277bdd-a2ba-43b0-a186-5bda332b0439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.302210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6277bdd-a2ba-43b0-a186-5bda332b0439 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.308985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6277bdd-a2ba-43b0-a186-5bda332b0439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.308985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6277bdd-a2ba-43b0-a186-5bda332b0439 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.308985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6277bdd-a2ba-43b0-a186-5bda332b0439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.308985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6277bdd-a2ba-43b0-a186-5bda332b0439 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.319176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1223/2416] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:52 2022] PUT /identity/v3/projects/26e91c6957334581beb2200ba6f4ffc3/users/60401da2abf34807a3d7bb8cf4b82856/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:52.322995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4c4b3-1391-4641-a1ef-98492fbe6a8a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.389568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4c4b3-1391-4641-a1ef-98492fbe6a8a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.394261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd1340-ee1a-4f69-b70b-aacc6ee4bc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.394872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd1340-ee1a-4f69-b70b-aacc6ee4bc8b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.395149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd1340-ee1a-4f69-b70b-aacc6ee4bc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.395562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd1340-ee1a-4f69-b70b-aacc6ee4bc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.396263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd1340-ee1a-4f69-b70b-aacc6ee4bc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.397888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd1340-ee1a-4f69-b70b-aacc6ee4bc8b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.408985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1224/2417] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:49:52.412912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd1340-ee1a-4f69-b70b-aacc6ee4bc8b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.424241 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd1340-ee1a-4f69-b70b-aacc6ee4bc8b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.430480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936f3f6-fe02-4e36-922f-2ab33d7f31c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.431451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936f3f6-fe02-4e36-922f-2ab33d7f31c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.431731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936f3f6-fe02-4e36-922f-2ab33d7f31c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.432003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936f3f6-fe02-4e36-922f-2ab33d7f31c7 admin admin] PATH_INFO: `/v3/projects/26e91c6957334581beb2200ba6f4ffc3/users/60401da2abf34807a3d7bb8cf4b82856/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.443194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936f3f6-fe02-4e36-922f-2ab33d7f31c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e91c6957334581beb2200ba6f4ffc3, user_id=60401da2abf34807a3d7bb8cf4b82856, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.447829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1194/2418] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:49:52 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:49:52.447984 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936f3f6-fe02-4e36-922f-2ab33d7f31c7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.450492 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6277bdd-a2ba-43b0-a186-5bda332b0439 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.468469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1225/2419] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:52 2022] PUT /identity/v3/projects/26e91c6957334581beb2200ba6f4ffc3/users/60401da2abf34807a3d7bb8cf4b82856/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:52.474524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936f3f6-fe02-4e36-922f-2ab33d7f31c7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.476225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0936f3f6-fe02-4e36-922f-2ab33d7f31c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:52.492043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:52.531317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6277bdd-a2ba-43b0-a186-5bda332b0439 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.539484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936f3f6-fe02-4e36-922f-2ab33d7f31c7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.543337 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a9904-1d53-4dc9-9fdc-0a0f59eda6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.544247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a9904-1d53-4dc9-9fdc-0a0f59eda6ce service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.546632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a9904-1d53-4dc9-9fdc-0a0f59eda6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.546632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a9904-1d53-4dc9-9fdc-0a0f59eda6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.547245 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ee8a9904-1d53-4dc9-9fdc-0a0f59eda6ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:52.592383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4ff2a-0b58-40ef-9077-0e82907aa7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.593504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4ff2a-0b58-40ef-9077-0e82907aa7b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.593778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4ff2a-0b58-40ef-9077-0e82907aa7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.594048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4ff2a-0b58-40ef-9077-0e82907aa7b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.594696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4ff2a-0b58-40ef-9077-0e82907aa7b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.596601 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4ff2a-0b58-40ef-9077-0e82907aa7b5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.611830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a9904-1d53-4dc9-9fdc-0a0f59eda6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.613347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a9904-1d53-4dc9-9fdc-0a0f59eda6ce service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.623375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1226/2420] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:52.624638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ce730-2913-4463-b76c-d03cf92f00f6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.624922 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ce730-2913-4463-b76c-d03cf92f00f6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.625196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ce730-2913-4463-b76c-d03cf92f00f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.652133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6b8ce730-2913-4463-b76c-d03cf92f00f6 None None] MFA Rules not processed for user `60401da2abf34807a3d7bb8cf4b82856`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:52.653350 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6b8ce730-2913-4463-b76c-d03cf92f00f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:52.677726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1195/2421] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:49:52 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:52.681776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4ff2a-0b58-40ef-9077-0e82907aa7b5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.692509 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4ff2a-0b58-40ef-9077-0e82907aa7b5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.697703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b006b87-9b16-4272-b58c-302195a72421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.698475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b006b87-9b16-4272-b58c-302195a72421 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.698857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b006b87-9b16-4272-b58c-302195a72421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.699233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b006b87-9b16-4272-b58c-302195a72421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.700000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b006b87-9b16-4272-b58c-302195a72421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.701720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b006b87-9b16-4272-b58c-302195a72421 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.718567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1196/2422] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:52.727810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b006b87-9b16-4272-b58c-302195a72421 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.738680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b006b87-9b16-4272-b58c-302195a72421 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.741199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1227/2423] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:52.744200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f548056-2c1e-4b85-af95-69095dab07d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.744984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f548056-2c1e-4b85-af95-69095dab07d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.746470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f548056-2c1e-4b85-af95-69095dab07d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.746762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f548056-2c1e-4b85-af95-69095dab07d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.747441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f548056-2c1e-4b85-af95-69095dab07d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.749102 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f548056-2c1e-4b85-af95-69095dab07d2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.753708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ce730-2913-4463-b76c-d03cf92f00f6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.755722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6b8ce730-2913-4463-b76c-d03cf92f00f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:52.761079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1197/2424] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:49:52.765290 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f548056-2c1e-4b85-af95-69095dab07d2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:52.775451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f548056-2c1e-4b85-af95-69095dab07d2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.783990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9b891-9adf-448b-8df3-e52840950e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.784664 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9b891-9adf-448b-8df3-e52840950e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.785003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9b891-9adf-448b-8df3-e52840950e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.785341 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9b891-9adf-448b-8df3-e52840950e58 admin admin] PATH_INFO: `/v3/projects/e8bd2ab952674d2a9f150200a30645bc/users/953e00eb58284efea3c6870bd9aca0ca/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.804929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9b891-9adf-448b-8df3-e52840950e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8bd2ab952674d2a9f150200a30645bc, user_id=953e00eb58284efea3c6870bd9aca0ca, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.808447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9b891-9adf-448b-8df3-e52840950e58 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.832097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:52.836332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ce730-2913-4463-b76c-d03cf92f00f6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:52.840827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bed725-5184-4c7a-a261-e5ba3393a944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:52.841438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bed725-5184-4c7a-a261-e5ba3393a944 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.841714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bed725-5184-4c7a-a261-e5ba3393a944 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.841986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bed725-5184-4c7a-a261-e5ba3393a944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.851842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bed725-5184-4c7a-a261-e5ba3393a944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:52.853203 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bed725-5184-4c7a-a261-e5ba3393a944 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:52.863098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1228/2425] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:49:52.940142 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1198/2426] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:52 2022] PUT /identity/v3/projects/e8bd2ab952674d2a9f150200a30645bc/users/953e00eb58284efea3c6870bd9aca0ca/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:52.950944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e18b7f-b6a3-4eb3-a33c-379a1f7d27d3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:52.955045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e18b7f-b6a3-4eb3-a33c-379a1f7d27d3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:52.955045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e18b7f-b6a3-4eb3-a33c-379a1f7d27d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:52.980416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f6e18b7f-b6a3-4eb3-a33c-379a1f7d27d3 None None] MFA Rules not processed for user `953e00eb58284efea3c6870bd9aca0ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:52.981587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f6e18b7f-b6a3-4eb3-a33c-379a1f7d27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:53.055209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1229/2427] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:53.059832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9b891-9adf-448b-8df3-e52840950e58 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:53.096609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:53.113440 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e18b7f-b6a3-4eb3-a33c-379a1f7d27d3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:53.130293 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9b891-9adf-448b-8df3-e52840950e58 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:53.135698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798beee-33f2-4170-88a9-9e9e35adbf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:53.136309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798beee-33f2-4170-88a9-9e9e35adbf99 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:53.136755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798beee-33f2-4170-88a9-9e9e35adbf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:53.137034 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798beee-33f2-4170-88a9-9e9e35adbf99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:53.137662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f798beee-33f2-4170-88a9-9e9e35adbf99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:53.139617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f798beee-33f2-4170-88a9-9e9e35adbf99 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:53.180012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e18b7f-b6a3-4eb3-a33c-379a1f7d27d3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:53.184893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a25311-f884-4815-9a21-586190d95649 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:53.185511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a25311-f884-4815-9a21-586190d95649 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:53.186172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a25311-f884-4815-9a21-586190d95649 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:53.186172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a25311-f884-4815-9a21-586190d95649 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:53.187901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-22a25311-f884-4815-9a21-586190d95649 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:53.228176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1199/2428] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:49:53 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:49:53.229306 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798beee-33f2-4170-88a9-9e9e35adbf99 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:53.261781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a25311-f884-4815-9a21-586190d95649 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:53.263270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a25311-f884-4815-9a21-586190d95649 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:53.279051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1230/2429] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:53.303717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798beee-33f2-4170-88a9-9e9e35adbf99 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:53.311445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae36ac-63c1-44bc-8060-2a189f76b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:53.312585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae36ac-63c1-44bc-8060-2a189f76b452 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:53.312865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae36ac-63c1-44bc-8060-2a189f76b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:53.313166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae36ac-63c1-44bc-8060-2a189f76b452 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:53.313905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae36ac-63c1-44bc-8060-2a189f76b452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:53.316402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae36ac-63c1-44bc-8060-2a189f76b452 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:53.405436 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1200/2430] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:49:53 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:49:53.409207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a25311-f884-4815-9a21-586190d95649 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:53.420453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a25311-f884-4815-9a21-586190d95649 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:53.427570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49957799-2a60-4cd8-bf52-4141d186b971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:53.428256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49957799-2a60-4cd8-bf52-4141d186b971 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:53.428602 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49957799-2a60-4cd8-bf52-4141d186b971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:53.428962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49957799-2a60-4cd8-bf52-4141d186b971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:53.429694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49957799-2a60-4cd8-bf52-4141d186b971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:53.431551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49957799-2a60-4cd8-bf52-4141d186b971 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:53.447243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1231/2431] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:53.458002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae36ac-63c1-44bc-8060-2a189f76b452 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:53.468045 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae36ac-63c1-44bc-8060-2a189f76b452 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:53.472713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222203b9-e820-455d-a4d5-5c06ca572ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:53.473402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222203b9-e820-455d-a4d5-5c06ca572ece admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:53.473693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222203b9-e820-455d-a4d5-5c06ca572ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:53.473956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222203b9-e820-455d-a4d5-5c06ca572ece admin admin] PATH_INFO: `/v3/projects/3e75ec26a86e4e1194e38d0bf704bf14/users/f86646cfb40a40da988df1f7fd71b0e4/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:53.490197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222203b9-e820-455d-a4d5-5c06ca572ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e75ec26a86e4e1194e38d0bf704bf14, user_id=f86646cfb40a40da988df1f7fd71b0e4, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:53.492836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222203b9-e820-455d-a4d5-5c06ca572ece admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:53.603358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1201/2432] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:53 2022] PUT /identity/v3/projects/3e75ec26a86e4e1194e38d0bf704bf14/users/f86646cfb40a40da988df1f7fd71b0e4/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:53.607499 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49957799-2a60-4cd8-bf52-4141d186b971 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:53.617251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49957799-2a60-4cd8-bf52-4141d186b971 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:53.637480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:53.674699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbf787-3edb-42dd-ba26-d06d6fd571bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:53.675789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbf787-3edb-42dd-ba26-d06d6fd571bb admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:53.676268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbf787-3edb-42dd-ba26-d06d6fd571bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:53.676735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbf787-3edb-42dd-ba26-d06d6fd571bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:53.677563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbf787-3edb-42dd-ba26-d06d6fd571bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:53.679387 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbf787-3edb-42dd-ba26-d06d6fd571bb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:53.691172 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1232/2433] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:53.695527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222203b9-e820-455d-a4d5-5c06ca572ece admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:53.706194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222203b9-e820-455d-a4d5-5c06ca572ece admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:53.715864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1b7c5-0ae1-4b27-9671-34239991f04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:53.716644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1b7c5-0ae1-4b27-9671-34239991f04e admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:53.717023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1b7c5-0ae1-4b27-9671-34239991f04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:53.717394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1b7c5-0ae1-4b27-9671-34239991f04e admin admin] PATH_INFO: `/v3/projects/3e75ec26a86e4e1194e38d0bf704bf14/users/f86646cfb40a40da988df1f7fd71b0e4/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:53.719690 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1b7c5-0ae1-4b27-9671-34239991f04e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e75ec26a86e4e1194e38d0bf704bf14, user_id=f86646cfb40a40da988df1f7fd71b0e4, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:53.722495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1b7c5-0ae1-4b27-9671-34239991f04e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:53.746830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1202/2434] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:53 2022] PUT /identity/v3/projects/3e75ec26a86e4e1194e38d0bf704bf14/users/f86646cfb40a40da988df1f7fd71b0e4/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:53.750699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d1730-b79d-44e4-9b11-b4e53d414a75 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:53.751090 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d1730-b79d-44e4-9b11-b4e53d414a75 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:53.751413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d1730-b79d-44e4-9b11-b4e53d414a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:53.785243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-659d1730-b79d-44e4-9b11-b4e53d414a75 None None] MFA Rules not processed for user `f86646cfb40a40da988df1f7fd71b0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:53.789552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-659d1730-b79d-44e4-9b11-b4e53d414a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:53.877457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1233/2435] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:53.890080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1b7c5-0ae1-4b27-9671-34239991f04e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:53.892769 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7eb1b7c5-0ae1-4b27-9671-34239991f04e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:53.957012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1b7c5-0ae1-4b27-9671-34239991f04e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:53.962154 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc75de8-72b7-4807-aecb-2faa75ea13ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:53.963061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc75de8-72b7-4807-aecb-2faa75ea13ae service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:53.963394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc75de8-72b7-4807-aecb-2faa75ea13ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:53.963720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc75de8-72b7-4807-aecb-2faa75ea13ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:53.982250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc75de8-72b7-4807-aecb-2faa75ea13ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:53.984229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc75de8-72b7-4807-aecb-2faa75ea13ae service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:53.994811 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1203/2436] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:49:54.028847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d1730-b79d-44e4-9b11-b4e53d414a75 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:54.097660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d1730-b79d-44e4-9b11-b4e53d414a75 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:54.102616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:54.103463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:54.103745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:54.104023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] PATH_INFO: `/v3/users/953e00eb58284efea3c6870bd9aca0ca` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:54.105196 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953e00eb58284efea3c6870bd9aca0ca)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:54.107163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:54.297134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953e00eb58284efea3c6870bd9aca0ca'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:54.298447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:54.311290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953e00eb58284efea3c6870bd9aca0ca'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:54.312008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953e00eb58284efea3c6870bd9aca0ca'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:54.384872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953e00eb58284efea3c6870bd9aca0ca'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:54.385569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953e00eb58284efea3c6870bd9aca0ca'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:54.399724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1234/2437] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:49:54 2022] DELETE /identity/v3/users/953e00eb58284efea3c6870bd9aca0ca => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:54.404242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ae00c-dcc4-4901-a911-88851d1014c0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:54.404722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ae00c-dcc4-4901-a911-88851d1014c0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:54.405055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ae00c-dcc4-4901-a911-88851d1014c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:54.427508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-050ae00c-dcc4-4901-a911-88851d1014c0 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:54.429096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-050ae00c-dcc4-4901-a911-88851d1014c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:54.435080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:54.498180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1204/2438] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:54.506565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:54.514293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:54.575728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5252a28-4625-4401-802c-73c2b105a061 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:54.582597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657555e9-c2c8-492b-982c-d6cb6a8a21c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:54.583223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657555e9-c2c8-492b-982c-d6cb6a8a21c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:54.583545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657555e9-c2c8-492b-982c-d6cb6a8a21c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:54.583823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657555e9-c2c8-492b-982c-d6cb6a8a21c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:54.595401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657555e9-c2c8-492b-982c-d6cb6a8a21c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:54.597006 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657555e9-c2c8-492b-982c-d6cb6a8a21c6 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:54.607551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1235/2439] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:55.093214 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ae00c-dcc4-4901-a911-88851d1014c0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:55.095428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-050ae00c-dcc4-4901-a911-88851d1014c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:55.110303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ae00c-dcc4-4901-a911-88851d1014c0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:55.115295 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:55.116035 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:55.116313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:55.116635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] PATH_INFO: `/v3/projects/e8bd2ab952674d2a9f150200a30645bc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:55.117768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8bd2ab952674d2a9f150200a30645bc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:55.120067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:55.284197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8bd2ab952674d2a9f150200a30645bc', 'request_id': 'req-546277e0-eac4-4169-8078-e557df86cc1d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:55.290519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8bd2ab952674d2a9f150200a30645bc', 'request_id': 'req-546277e0-eac4-4169-8078-e557df86cc1d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:55.420785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8bd2ab952674d2a9f150200a30645bc', 'request_id': 'req-546277e0-eac4-4169-8078-e557df86cc1d'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:55.433279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] The token cache is being invalidate because project e8bd2ab952674d2a9f150200a30645bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:49:55.433803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:55.436043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:55.440285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1205/2440] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:49:55 2022] DELETE /identity/v3/projects/e8bd2ab952674d2a9f150200a30645bc => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:55.444597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657555e9-c2c8-492b-982c-d6cb6a8a21c6 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:55.454684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657555e9-c2c8-492b-982c-d6cb6a8a21c6 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:55.515401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:55.516142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:55.518541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:55.519215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] PATH_INFO: `/v3/users/f86646cfb40a40da988df1f7fd71b0e4` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:55.520024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86646cfb40a40da988df1f7fd71b0e4)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:55.522020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:55.727527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86646cfb40a40da988df1f7fd71b0e4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:55.753716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86646cfb40a40da988df1f7fd71b0e4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:55.754563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86646cfb40a40da988df1f7fd71b0e4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:55.761727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:55.866697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86646cfb40a40da988df1f7fd71b0e4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:55.867324 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86646cfb40a40da988df1f7fd71b0e4'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:55.894408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1236/2441] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:49:55 2022] DELETE /identity/v3/users/f86646cfb40a40da988df1f7fd71b0e4 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:56.408281 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:56.410240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:56.438909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546277e0-eac4-4169-8078-e557df86cc1d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:56.506304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:56.507726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:56.508300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:56.508817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] PATH_INFO: `/v3/projects/3e75ec26a86e4e1194e38d0bf704bf14` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:56.510990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e75ec26a86e4e1194e38d0bf704bf14)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:56.513809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:56.624913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:56.633639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e75ec26a86e4e1194e38d0bf704bf14', 'request_id': 'req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:56.640721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e75ec26a86e4e1194e38d0bf704bf14', 'request_id': 'req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:56.701273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:56.702909 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:56.712775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e75ec26a86e4e1194e38d0bf704bf14', 'request_id': 'req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:56.716276 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b753-6dee-4cb8-b474-ec1843423968 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:56.725420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] The token cache is being invalidate because project 3e75ec26a86e4e1194e38d0bf704bf14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:49:56.725717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:56.726418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-c0e3b2d1-0cb1-4dbb-aff0-b34d69ba12f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:56.727922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1206/2442] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:49:56 2022] DELETE /identity/v3/projects/3e75ec26a86e4e1194e38d0bf704bf14 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:56.732799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd8d0b-3c14-43f7-a2cc-7efd0d3bfe46 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:56.733082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd8d0b-3c14-43f7-a2cc-7efd0d3bfe46 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:56.733841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd8d0b-3c14-43f7-a2cc-7efd0d3bfe46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:56.753771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-13dd8d0b-3c14-43f7-a2cc-7efd0d3bfe46 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:56.755048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-13dd8d0b-3c14-43f7-a2cc-7efd0d3bfe46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:56.774790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:56.775422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:56.775698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:56.775977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] PATH_INFO: `/v3/users/537981b1b3a14cb4b10f336ef0362358` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:56.777089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] RBAC: Authorizing `identity:delete_user(user_id=537981b1b3a14cb4b10f336ef0362358)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:56.779027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:56.816707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1207/2443] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:56.824429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df25947-6c9a-43ea-8f6f-69308eb3b4f2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:56.824688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df25947-6c9a-43ea-8f6f-69308eb3b4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:56.824971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df25947-6c9a-43ea-8f6f-69308eb3b4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:56.844735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0df25947-6c9a-43ea-8f6f-69308eb3b4f2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:56.845954 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0df25947-6c9a-43ea-8f6f-69308eb3b4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:56.854227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1208/2444] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:56.859212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df25947-6c9a-43ea-8f6f-69308eb3b4f2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:56.870870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df25947-6c9a-43ea-8f6f-69308eb3b4f2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:56.879072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19553b2-66fa-4525-a8f2-a1ed4b09c8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:56.879794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19553b2-66fa-4525-a8f2-a1ed4b09c8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:56.880068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19553b2-66fa-4525-a8f2-a1ed4b09c8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:56.880332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19553b2-66fa-4525-a8f2-a1ed4b09c8e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:56.880969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19553b2-66fa-4525-a8f2-a1ed4b09c8e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:49:56.881268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19553b2-66fa-4525-a8f2-a1ed4b09c8e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:56.882935 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19553b2-66fa-4525-a8f2-a1ed4b09c8e0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:56.902643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1209/2445] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:49:56.907724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19553b2-66fa-4525-a8f2-a1ed4b09c8e0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:56.913409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19553b2-66fa-4525-a8f2-a1ed4b09c8e0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:56.921434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fceb3-5685-4f6a-9bcb-1e06abec8adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:56.921434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fceb3-5685-4f6a-9bcb-1e06abec8adf admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:56.921434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fceb3-5685-4f6a-9bcb-1e06abec8adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:56.921434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fceb3-5685-4f6a-9bcb-1e06abec8adf admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:56.921761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727fceb3-5685-4f6a-9bcb-1e06abec8adf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:56.922629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727fceb3-5685-4f6a-9bcb-1e06abec8adf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:56.971243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '537981b1b3a14cb4b10f336ef0362358'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:56.983885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '537981b1b3a14cb4b10f336ef0362358'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:56.984586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '537981b1b3a14cb4b10f336ef0362358'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:57.009799 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1210/2446] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:49:56 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:57.013490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fceb3-5685-4f6a-9bcb-1e06abec8adf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:57.017343 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-727fceb3-5685-4f6a-9bcb-1e06abec8adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:57.080903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fceb3-5685-4f6a-9bcb-1e06abec8adf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:57.085764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4187b5f-4b2a-4654-bdbc-1682b298a9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:57.086394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4187b5f-4b2a-4654-bdbc-1682b298a9c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.086676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4187b5f-4b2a-4654-bdbc-1682b298a9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.086949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4187b5f-4b2a-4654-bdbc-1682b298a9c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.087825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4187b5f-4b2a-4654-bdbc-1682b298a9c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:57.089744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4187b5f-4b2a-4654-bdbc-1682b298a9c5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:57.100000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '537981b1b3a14cb4b10f336ef0362358'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:57.104293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '537981b1b3a14cb4b10f336ef0362358'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:57.114057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1237/2447] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:49:56 2022] DELETE /identity/v3/users/537981b1b3a14cb4b10f336ef0362358 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:57.192980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1211/2448] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:49:57 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:49:57.200824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:57.268454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef50549-43af-4c05-9966-28d880aa632e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:57.273396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78baf4-4d51-496b-bffa-0cf7872aaca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:57.274011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78baf4-4d51-496b-bffa-0cf7872aaca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.274330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78baf4-4d51-496b-bffa-0cf7872aaca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.274606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78baf4-4d51-496b-bffa-0cf7872aaca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.275293 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78baf4-4d51-496b-bffa-0cf7872aaca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:57.276880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78baf4-4d51-496b-bffa-0cf7872aaca5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:57.289485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1238/2449] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:57.293508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4187b5f-4b2a-4654-bdbc-1682b298a9c5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:57.304448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4187b5f-4b2a-4654-bdbc-1682b298a9c5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:57.310859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d85da-e686-4fed-af69-a17f952cd10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:57.310859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d85da-e686-4fed-af69-a17f952cd10d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.310859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d85da-e686-4fed-af69-a17f952cd10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.310859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d85da-e686-4fed-af69-a17f952cd10d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.311515 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d85da-e686-4fed-af69-a17f952cd10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:57.313094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d85da-e686-4fed-af69-a17f952cd10d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:57.324428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1212/2450] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:49:57.328538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78baf4-4d51-496b-bffa-0cf7872aaca5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:57.333803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:57.339509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78baf4-4d51-496b-bffa-0cf7872aaca5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:57.344724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513ff2f-9fae-49ea-aad3-29924de95d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:57.345368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513ff2f-9fae-49ea-aad3-29924de95d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.345641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513ff2f-9fae-49ea-aad3-29924de95d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.345908 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513ff2f-9fae-49ea-aad3-29924de95d44 admin admin] PATH_INFO: `/v3/projects/f7248edf24e34094ab2dfd803073241d/users/8b6f0b6242884bde96d3cb0f9719ffcc/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.360847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513ff2f-9fae-49ea-aad3-29924de95d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7248edf24e34094ab2dfd803073241d, user_id=8b6f0b6242884bde96d3cb0f9719ffcc, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:57.363508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513ff2f-9fae-49ea-aad3-29924de95d44 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:57.375233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d85da-e686-4fed-af69-a17f952cd10d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:57.376940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-663d85da-e686-4fed-af69-a17f952cd10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:57.385957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:57.391948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d85da-e686-4fed-af69-a17f952cd10d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:57.397130 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:57.397938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.398336 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.398509 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] PATH_INFO: `/v3/projects/7efd5ee13f094656992497528d761f2c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.399637 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7efd5ee13f094656992497528d761f2c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:57.401572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:57.473506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1239/2451] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:57 2022] PUT /identity/v3/projects/f7248edf24e34094ab2dfd803073241d/users/8b6f0b6242884bde96d3cb0f9719ffcc/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:57.488183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64525dc-03b7-43d3-924f-b670fb9ef3de None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.488473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64525dc-03b7-43d3-924f-b670fb9ef3de None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.488782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64525dc-03b7-43d3-924f-b670fb9ef3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.518585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-d64525dc-03b7-43d3-924f-b670fb9ef3de None None] MFA Rules not processed for user `8b6f0b6242884bde96d3cb0f9719ffcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:57.519747 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d64525dc-03b7-43d3-924f-b670fb9ef3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:57.550620 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7efd5ee13f094656992497528d761f2c', 'request_id': 'req-dddf1272-a815-4934-9ef7-b005259e3470'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:57.559163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7efd5ee13f094656992497528d761f2c', 'request_id': 'req-dddf1272-a815-4934-9ef7-b005259e3470'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:57.592331 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1240/2452] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:57.597375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64525dc-03b7-43d3-924f-b670fb9ef3de None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:57.660163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64525dc-03b7-43d3-924f-b670fb9ef3de None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:57.689713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7efd5ee13f094656992497528d761f2c', 'request_id': 'req-dddf1272-a815-4934-9ef7-b005259e3470'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:57.701902 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] The token cache is being invalidate because project 7efd5ee13f094656992497528d761f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:49:57.702759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:57.703390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-dddf1272-a815-4934-9ef7-b005259e3470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:49:57.704935 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1213/2453] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:49:57 2022] DELETE /identity/v3/projects/7efd5ee13f094656992497528d761f2c => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:57.709789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43348cf4-0ee8-425b-97f9-8a1bc56217f0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.714398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43348cf4-0ee8-425b-97f9-8a1bc56217f0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.714694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43348cf4-0ee8-425b-97f9-8a1bc56217f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.729903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1890d4-d4e3-4451-9c8d-c8143bd12479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:57.731202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1890d4-d4e3-4451-9c8d-c8143bd12479 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.731476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1890d4-d4e3-4451-9c8d-c8143bd12479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.731745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1890d4-d4e3-4451-9c8d-c8143bd12479 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.732828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1890d4-d4e3-4451-9c8d-c8143bd12479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:57.736713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1890d4-d4e3-4451-9c8d-c8143bd12479 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:57.737888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-43348cf4-0ee8-425b-97f9-8a1bc56217f0 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:57.741085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-43348cf4-0ee8-425b-97f9-8a1bc56217f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:57.753140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1214/2454] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:57.761845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb5ec7-f382-4cf8-a814-1651a190115d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.762255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb5ec7-f382-4cf8-a814-1651a190115d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.762603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb5ec7-f382-4cf8-a814-1651a190115d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.784027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5deb5ec7-f382-4cf8-a814-1651a190115d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:57.785247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5deb5ec7-f382-4cf8-a814-1651a190115d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:57.793409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1215/2455] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:57.797568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb5ec7-f382-4cf8-a814-1651a190115d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:57.806807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb5ec7-f382-4cf8-a814-1651a190115d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:57.812022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9530de-4df1-4d64-b4e0-e62a1920fc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:57.812651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9530de-4df1-4d64-b4e0-e62a1920fc05 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.816195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9530de-4df1-4d64-b4e0-e62a1920fc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.816195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9530de-4df1-4d64-b4e0-e62a1920fc05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.816195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9530de-4df1-4d64-b4e0-e62a1920fc05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:49:57.816195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9530de-4df1-4d64-b4e0-e62a1920fc05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:57.816195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9530de-4df1-4d64-b4e0-e62a1920fc05 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:57.827075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:57.831786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1216/2456] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:49:57.832797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9530de-4df1-4d64-b4e0-e62a1920fc05 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:57.834681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1241/2457] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:49:57 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:49:57.838354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1890d4-d4e3-4451-9c8d-c8143bd12479 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:57.840133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9a1890d4-d4e3-4451-9c8d-c8143bd12479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:57.842738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9530de-4df1-4d64-b4e0-e62a1920fc05 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:57.911237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7affd0-d9a4-4698-b2bc-3cd14d159d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:57.911929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7affd0-d9a4-4698-b2bc-3cd14d159d19 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.912125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7affd0-d9a4-4698-b2bc-3cd14d159d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.912396 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7affd0-d9a4-4698-b2bc-3cd14d159d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.913023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7affd0-d9a4-4698-b2bc-3cd14d159d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:57.913964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1890d4-d4e3-4451-9c8d-c8143bd12479 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:57.914940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7affd0-d9a4-4698-b2bc-3cd14d159d19 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:57.918789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a75500-db31-46f5-9f90-850a2ba4f4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:57.919390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a75500-db31-46f5-9f90-850a2ba4f4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:57.919670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a75500-db31-46f5-9f90-850a2ba4f4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:57.919943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a75500-db31-46f5-9f90-850a2ba4f4d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:57.920597 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a75500-db31-46f5-9f90-850a2ba4f4d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:57.922571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a75500-db31-46f5-9f90-850a2ba4f4d9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.004086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1242/2458] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:49:57 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:49:58.008729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a75500-db31-46f5-9f90-850a2ba4f4d9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.009224 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1217/2459] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:49:57 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:58.016596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7affd0-d9a4-4698-b2bc-3cd14d159d19 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.019672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a75500-db31-46f5-9f90-850a2ba4f4d9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.064970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:58.082951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7affd0-d9a4-4698-b2bc-3cd14d159d19 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.089108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4b9e7-752d-4028-9a4c-2f594c7248a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.089914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4b9e7-752d-4028-9a4c-2f594c7248a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.090222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4b9e7-752d-4028-9a4c-2f594c7248a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.090501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4b9e7-752d-4028-9a4c-2f594c7248a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.090927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76957401-b0cf-484c-86e5-1e62229d4a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.091129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4b9e7-752d-4028-9a4c-2f594c7248a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.091498 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76957401-b0cf-484c-86e5-1e62229d4a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.091774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76957401-b0cf-484c-86e5-1e62229d4a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.092048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76957401-b0cf-484c-86e5-1e62229d4a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.092720 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76957401-b0cf-484c-86e5-1e62229d4a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.093615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4b9e7-752d-4028-9a4c-2f594c7248a9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.094935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76957401-b0cf-484c-86e5-1e62229d4a01 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.107090 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1243/2460] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:58.111028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76957401-b0cf-484c-86e5-1e62229d4a01 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.120976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76957401-b0cf-484c-86e5-1e62229d4a01 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.126325 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fecab8-c5d5-4325-8387-26368a6fe331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.126938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fecab8-c5d5-4325-8387-26368a6fe331 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.127280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fecab8-c5d5-4325-8387-26368a6fe331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.127545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fecab8-c5d5-4325-8387-26368a6fe331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.128175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fecab8-c5d5-4325-8387-26368a6fe331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.129786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fecab8-c5d5-4325-8387-26368a6fe331 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.139573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1244/2461] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:49:58.144047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fecab8-c5d5-4325-8387-26368a6fe331 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.153458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fecab8-c5d5-4325-8387-26368a6fe331 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.158539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8bfef-a962-49b3-a2b2-e5c84205cd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.159152 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8bfef-a962-49b3-a2b2-e5c84205cd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.159485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8bfef-a962-49b3-a2b2-e5c84205cd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.159744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8bfef-a962-49b3-a2b2-e5c84205cd5a admin admin] PATH_INFO: `/v3/projects/dfd3423a51df420aa5e9323f8bafba08/users/8a18e15ca9af46b3abb6ba972320bb65/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.174571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8bfef-a962-49b3-a2b2-e5c84205cd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd3423a51df420aa5e9323f8bafba08, user_id=8a18e15ca9af46b3abb6ba972320bb65, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.177197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8bfef-a962-49b3-a2b2-e5c84205cd5a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.193430 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1218/2462] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:49:58 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:49:58.197452 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4b9e7-752d-4028-9a4c-2f594c7248a9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.207690 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4b9e7-752d-4028-9a4c-2f594c7248a9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.213254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0d863-3fc4-4f86-b2f9-3fe11a8ddaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.213875 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0d863-3fc4-4f86-b2f9-3fe11a8ddaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.214173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0d863-3fc4-4f86-b2f9-3fe11a8ddaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.214754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0d863-3fc4-4f86-b2f9-3fe11a8ddaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.215419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0d863-3fc4-4f86-b2f9-3fe11a8ddaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.217003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0d863-3fc4-4f86-b2f9-3fe11a8ddaa6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.226982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1219/2463] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:49:58.238784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0d863-3fc4-4f86-b2f9-3fe11a8ddaa6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.248601 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0d863-3fc4-4f86-b2f9-3fe11a8ddaa6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.253870 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300462a-09f8-4287-9869-c59f83490610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.258375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300462a-09f8-4287-9869-c59f83490610 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.258375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300462a-09f8-4287-9869-c59f83490610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.258375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300462a-09f8-4287-9869-c59f83490610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.258375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300462a-09f8-4287-9869-c59f83490610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.258375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300462a-09f8-4287-9869-c59f83490610 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.267503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1220/2464] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:49:58.271378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300462a-09f8-4287-9869-c59f83490610 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.280583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300462a-09f8-4287-9869-c59f83490610 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.285234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e6822-07fe-4496-9da6-b5c6711e73d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.285893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e6822-07fe-4496-9da6-b5c6711e73d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.286124 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e6822-07fe-4496-9da6-b5c6711e73d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.286497 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e6822-07fe-4496-9da6-b5c6711e73d2 admin admin] PATH_INFO: `/v3/projects/7b43ccfbf77d42dcbab0b85aceeb4324/users/dc6572065fca458299c5485f5ae28661/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.301666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e6822-07fe-4496-9da6-b5c6711e73d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b43ccfbf77d42dcbab0b85aceeb4324, user_id=dc6572065fca458299c5485f5ae28661, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.304361 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e6822-07fe-4496-9da6-b5c6711e73d2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.305907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1245/2465] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:58 2022] PUT /identity/v3/projects/dfd3423a51df420aa5e9323f8bafba08/users/8a18e15ca9af46b3abb6ba972320bb65/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:58.317757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b45f-5fb5-475e-a819-f525516b320f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.318061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b45f-5fb5-475e-a819-f525516b320f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.318410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b45f-5fb5-475e-a819-f525516b320f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.345606 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6cc3b45f-5fb5-475e-a819-f525516b320f None None] MFA Rules not processed for user `8a18e15ca9af46b3abb6ba972320bb65`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:58.346885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6cc3b45f-5fb5-475e-a819-f525516b320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:58.391631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1221/2466] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:58 2022] PUT /identity/v3/projects/7b43ccfbf77d42dcbab0b85aceeb4324/users/dc6572065fca458299c5485f5ae28661/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:49:58.394126 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:58.403017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3b905-0c56-4c2f-9252-0f8de96bc4f0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.403350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3b905-0c56-4c2f-9252-0f8de96bc4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.403846 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3b905-0c56-4c2f-9252-0f8de96bc4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.419074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1246/2467] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:58.423996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b45f-5fb5-475e-a819-f525516b320f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.434853 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3ac3b905-0c56-4c2f-9252-0f8de96bc4f0 None None] MFA Rules not processed for user `dc6572065fca458299c5485f5ae28661`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:58.443923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3ac3b905-0c56-4c2f-9252-0f8de96bc4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:58.489790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b45f-5fb5-475e-a819-f525516b320f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.509928 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1222/2468] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:49:58.525630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3b905-0c56-4c2f-9252-0f8de96bc4f0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.527039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3ac3b905-0c56-4c2f-9252-0f8de96bc4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:58.552935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf8092-6bd3-43f1-9026-b7090212352b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.555202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf8092-6bd3-43f1-9026-b7090212352b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.555202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf8092-6bd3-43f1-9026-b7090212352b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.555202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf8092-6bd3-43f1-9026-b7090212352b admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.555202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf8092-6bd3-43f1-9026-b7090212352b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.556839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf8092-6bd3-43f1-9026-b7090212352b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.607136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3b905-0c56-4c2f-9252-0f8de96bc4f0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.613433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc82fb-5465-4ec1-90d7-834a4672a157 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.614027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc82fb-5465-4ec1-90d7-834a4672a157 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.614332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc82fb-5465-4ec1-90d7-834a4672a157 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.614601 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc82fb-5465-4ec1-90d7-834a4672a157 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.622213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc82fb-5465-4ec1-90d7-834a4672a157 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.623770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc82fb-5465-4ec1-90d7-834a4672a157 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.632358 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1223/2469] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:49:58.645096 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1247/2470] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:49:58 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:58.649063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc82fb-5465-4ec1-90d7-834a4672a157 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.654409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:58.660792 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc82fb-5465-4ec1-90d7-834a4672a157 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.720178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dded4-7e68-402e-93bb-350e7524fa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.720780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dded4-7e68-402e-93bb-350e7524fa79 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.721060 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dded4-7e68-402e-93bb-350e7524fa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.721332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dded4-7e68-402e-93bb-350e7524fa79 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.722026 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dded4-7e68-402e-93bb-350e7524fa79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.724001 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dded4-7e68-402e-93bb-350e7524fa79 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.826093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1224/2471] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:49:58 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:49:58.830044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf8092-6bd3-43f1-9026-b7090212352b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.840560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf8092-6bd3-43f1-9026-b7090212352b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.845882 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfeef2-d109-484e-93d6-9236ba529830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.846509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfeef2-d109-484e-93d6-9236ba529830 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.846850 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfeef2-d109-484e-93d6-9236ba529830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.849552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfeef2-d109-484e-93d6-9236ba529830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.849552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdfeef2-d109-484e-93d6-9236ba529830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.849552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdfeef2-d109-484e-93d6-9236ba529830 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.860541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1248/2472] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:58.864687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dded4-7e68-402e-93bb-350e7524fa79 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.874730 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dded4-7e68-402e-93bb-350e7524fa79 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:58.879455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834b898-8046-4bcd-a8ca-d49255e44bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:58.880088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834b898-8046-4bcd-a8ca-d49255e44bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:58.880364 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834b898-8046-4bcd-a8ca-d49255e44bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:58.880635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834b898-8046-4bcd-a8ca-d49255e44bb1 admin admin] PATH_INFO: `/v3/projects/713981f7dbdc4e5195cbf59e946e9d31/users/0c171dee265246cd8554c20f10025052/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:58.896739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834b898-8046-4bcd-a8ca-d49255e44bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=713981f7dbdc4e5195cbf59e946e9d31, user_id=0c171dee265246cd8554c20f10025052, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:58.899494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834b898-8046-4bcd-a8ca-d49255e44bb1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:58.974644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1225/2473] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:58 2022] PUT /identity/v3/projects/713981f7dbdc4e5195cbf59e946e9d31/users/0c171dee265246cd8554c20f10025052/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:58.978601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfeef2-d109-484e-93d6-9236ba529830 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:58.988361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfeef2-d109-484e-93d6-9236ba529830 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:59.043883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b5023-2e32-45fa-b6c1-050dea975595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:59.044415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b5023-2e32-45fa-b6c1-050dea975595 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:59.044724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b5023-2e32-45fa-b6c1-050dea975595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:59.045060 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b5023-2e32-45fa-b6c1-050dea975595 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:59.045697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b5023-2e32-45fa-b6c1-050dea975595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:59.047358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b5023-2e32-45fa-b6c1-050dea975595 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:59.049376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:49:59.057630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1249/2474] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:49:59.061965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834b898-8046-4bcd-a8ca-d49255e44bb1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:59.072639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834b898-8046-4bcd-a8ca-d49255e44bb1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:59.077729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328ed2a-c5aa-4ea0-88e2-3b91befb59fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:59.078394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328ed2a-c5aa-4ea0-88e2-3b91befb59fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:59.078670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328ed2a-c5aa-4ea0-88e2-3b91befb59fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:59.078937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328ed2a-c5aa-4ea0-88e2-3b91befb59fe admin admin] PATH_INFO: `/v3/projects/713981f7dbdc4e5195cbf59e946e9d31/users/0c171dee265246cd8554c20f10025052/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:59.080967 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328ed2a-c5aa-4ea0-88e2-3b91befb59fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=713981f7dbdc4e5195cbf59e946e9d31, user_id=0c171dee265246cd8554c20f10025052, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:59.083703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328ed2a-c5aa-4ea0-88e2-3b91befb59fe admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:59.094940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1226/2475] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:49:59 2022] PUT /identity/v3/projects/713981f7dbdc4e5195cbf59e946e9d31/users/0c171dee265246cd8554c20f10025052/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:49:59.105110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb89f-66e2-45ee-aea8-ecb7dbae65be None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:59.105398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb89f-66e2-45ee-aea8-ecb7dbae65be None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:59.105678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb89f-66e2-45ee-aea8-ecb7dbae65be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:59.132810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-5cefb89f-66e2-45ee-aea8-ecb7dbae65be None None] MFA Rules not processed for user `0c171dee265246cd8554c20f10025052`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:49:59.134012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5cefb89f-66e2-45ee-aea8-ecb7dbae65be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:59.199553 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1250/2476] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:49:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:49:59.211263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328ed2a-c5aa-4ea0-88e2-3b91befb59fe admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:49:59.212973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0328ed2a-c5aa-4ea0-88e2-3b91befb59fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:59.270426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328ed2a-c5aa-4ea0-88e2-3b91befb59fe admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:49:59.274223 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64482b69-6da2-4741-a3a4-18b51c4c299a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:49:59.274828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64482b69-6da2-4741-a3a4-18b51c4c299a service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:49:59.275109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64482b69-6da2-4741-a3a4-18b51c4c299a service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:49:59.275468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64482b69-6da2-4741-a3a4-18b51c4c299a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:49:59.277403 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-64482b69-6da2-4741-a3a4-18b51c4c299a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:49:59.339007 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64482b69-6da2-4741-a3a4-18b51c4c299a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:49:59.340427 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64482b69-6da2-4741-a3a4-18b51c4c299a service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:49:59.350054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1227/2477] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:49:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:00.076047 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb89f-66e2-45ee-aea8-ecb7dbae65be None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:00.077734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-5cefb89f-66e2-45ee-aea8-ecb7dbae65be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:00.144385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb89f-66e2-45ee-aea8-ecb7dbae65be None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:00.148925 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:00.153813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:00.153813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:00.153813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] PATH_INFO: `/v3/users/e8e3d3ed9b564e6c92835538cb9693a0` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:00.153813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e3d3ed9b564e6c92835538cb9693a0)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:00.153813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:00.370269 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e3d3ed9b564e6c92835538cb9693a0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:00.381958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e3d3ed9b564e6c92835538cb9693a0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:00.382646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e3d3ed9b564e6c92835538cb9693a0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:00.442384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e3d3ed9b564e6c92835538cb9693a0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:00.443008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e3d3ed9b564e6c92835538cb9693a0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:00.460147 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1251/2478] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:00 2022] DELETE /identity/v3/users/e8e3d3ed9b564e6c92835538cb9693a0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:00.464431 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93ef11-848a-4fd5-a0c9-792030bfe6d7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:00.464908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93ef11-848a-4fd5-a0c9-792030bfe6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:00.465015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93ef11-848a-4fd5-a0c9-792030bfe6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:00.484375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9e93ef11-848a-4fd5-a0c9-792030bfe6d7 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:00.485571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9e93ef11-848a-4fd5-a0c9-792030bfe6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:00.489589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:00.545189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1228/2479] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:00.554993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:00.556637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:00.562643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:00.617919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb4d8b-9a4f-44b5-b09a-85a98f43285f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:00.621457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf60d3-61a7-4014-b0b1-ea6c56aa7bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:00.622049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf60d3-61a7-4014-b0b1-ea6c56aa7bca service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:00.622354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf60d3-61a7-4014-b0b1-ea6c56aa7bca service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:00.622624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf60d3-61a7-4014-b0b1-ea6c56aa7bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:00.631338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf60d3-61a7-4014-b0b1-ea6c56aa7bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:00.632791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf60d3-61a7-4014-b0b1-ea6c56aa7bca service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:00.648121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1252/2480] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:01.088589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93ef11-848a-4fd5-a0c9-792030bfe6d7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:01.090067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9e93ef11-848a-4fd5-a0c9-792030bfe6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:01.108990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93ef11-848a-4fd5-a0c9-792030bfe6d7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:01.126652 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:01.127161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:01.127434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:01.127706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] PATH_INFO: `/v3/projects/b0b81dfe4d85414ea48e4e69b7a14a5e` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:01.128863 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b81dfe4d85414ea48e4e69b7a14a5e)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:01.130803 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:01.255801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b81dfe4d85414ea48e4e69b7a14a5e', 'request_id': 'req-5a156cb8-5980-4e06-a80f-fcf299995223'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:01.262217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b81dfe4d85414ea48e4e69b7a14a5e', 'request_id': 'req-5a156cb8-5980-4e06-a80f-fcf299995223'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:01.362486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b81dfe4d85414ea48e4e69b7a14a5e', 'request_id': 'req-5a156cb8-5980-4e06-a80f-fcf299995223'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:01.375098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] The token cache is being invalidate because project b0b81dfe4d85414ea48e4e69b7a14a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:01.375465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:01.376128 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:01.377640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1229/2481] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:01 2022] DELETE /identity/v3/projects/b0b81dfe4d85414ea48e4e69b7a14a5e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:01.385793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf60d3-61a7-4014-b0b1-ea6c56aa7bca service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:01.387403 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c0cf60d3-61a7-4014-b0b1-ea6c56aa7bca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:01.388704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:01.390898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:01.445329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf60d3-61a7-4014-b0b1-ea6c56aa7bca service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:01.454807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a156cb8-5980-4e06-a80f-fcf299995223 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:01.459498 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:01.460120 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:01.460398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:01.460673 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] PATH_INFO: `/v3/users/60401da2abf34807a3d7bb8cf4b82856` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:01.461804 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] RBAC: Authorizing `identity:delete_user(user_id=60401da2abf34807a3d7bb8cf4b82856)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:01.463742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:01.494048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e638ee-6ec9-4d5e-88cc-3b0ae840a22f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:01.494667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e638ee-6ec9-4d5e-88cc-3b0ae840a22f service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:01.494941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e638ee-6ec9-4d5e-88cc-3b0ae840a22f service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:01.495208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e638ee-6ec9-4d5e-88cc-3b0ae840a22f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:01.497049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-95e638ee-6ec9-4d5e-88cc-3b0ae840a22f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:01.554632 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e638ee-6ec9-4d5e-88cc-3b0ae840a22f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:01.556055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e638ee-6ec9-4d5e-88cc-3b0ae840a22f service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:01.565258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1253/2482] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:01.608356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:01.617647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60401da2abf34807a3d7bb8cf4b82856'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:01.618368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60401da2abf34807a3d7bb8cf4b82856'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:01.631015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60401da2abf34807a3d7bb8cf4b82856'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:01.729049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60401da2abf34807a3d7bb8cf4b82856'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:01.729712 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-52248056-27e2-4d9a-b935-4588dc0b685e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60401da2abf34807a3d7bb8cf4b82856'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:01.742578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1230/2483] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:01 2022] DELETE /identity/v3/users/60401da2abf34807a3d7bb8cf4b82856 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:02.175975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e638ee-6ec9-4d5e-88cc-3b0ae840a22f service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:02.189903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e638ee-6ec9-4d5e-88cc-3b0ae840a22f service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:02.203074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:02.248044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:02.248828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:02.249108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:02.249379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] PATH_INFO: `/v3/projects/26e91c6957334581beb2200ba6f4ffc3` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:02.251216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e91c6957334581beb2200ba6f4ffc3)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:02.253250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:02.392041 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e91c6957334581beb2200ba6f4ffc3', 'request_id': 'req-3b1c9e33-51bb-4410-b408-a07b106615b7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:02.398222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e91c6957334581beb2200ba6f4ffc3', 'request_id': 'req-3b1c9e33-51bb-4410-b408-a07b106615b7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:02.511672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e91c6957334581beb2200ba6f4ffc3', 'request_id': 'req-3b1c9e33-51bb-4410-b408-a07b106615b7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:02.523053 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] The token cache is being invalidate because project 26e91c6957334581beb2200ba6f4ffc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:02.523345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:02.523975 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-3b1c9e33-51bb-4410-b408-a07b106615b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:02.525454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1254/2484] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:02 2022] DELETE /identity/v3/projects/26e91c6957334581beb2200ba6f4ffc3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:02.530557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d33d44-bc83-48dd-aea0-7d25428df275 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:02.531347 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d33d44-bc83-48dd-aea0-7d25428df275 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:02.531621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d33d44-bc83-48dd-aea0-7d25428df275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:02.551557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-76d33d44-bc83-48dd-aea0-7d25428df275 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:02.552750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-76d33d44-bc83-48dd-aea0-7d25428df275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:02.611184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1231/2485] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:02.619201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c88b-9da2-4243-b51c-feb997a96fd6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:02.619490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c88b-9da2-4243-b51c-feb997a96fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:02.619769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4c88b-9da2-4243-b51c-feb997a96fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:02.639072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e8d4c88b-9da2-4243-b51c-feb997a96fd6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:02.640273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e8d4c88b-9da2-4243-b51c-feb997a96fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:02.648190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1255/2486] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:02.652362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d33d44-bc83-48dd-aea0-7d25428df275 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:02.662385 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d33d44-bc83-48dd-aea0-7d25428df275 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:02.666931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf700db-4194-4154-bddf-347a9c2e83f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:02.667518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf700db-4194-4154-bddf-347a9c2e83f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:02.667794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf700db-4194-4154-bddf-347a9c2e83f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:02.668062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf700db-4194-4154-bddf-347a9c2e83f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:02.668704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf700db-4194-4154-bddf-347a9c2e83f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:50:02.669005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf700db-4194-4154-bddf-347a9c2e83f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:02.670621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf700db-4194-4154-bddf-347a9c2e83f4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:02.682429 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1232/2487] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:50:02.686572 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4c88b-9da2-4243-b51c-feb997a96fd6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:02.696726 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4c88b-9da2-4243-b51c-feb997a96fd6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:02.701433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c576c2-38c9-47d8-a5cb-5b9cb65eab96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:02.702037 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c576c2-38c9-47d8-a5cb-5b9cb65eab96 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:02.702349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c576c2-38c9-47d8-a5cb-5b9cb65eab96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:02.703161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c576c2-38c9-47d8-a5cb-5b9cb65eab96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:02.703792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c576c2-38c9-47d8-a5cb-5b9cb65eab96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:02.705707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c576c2-38c9-47d8-a5cb-5b9cb65eab96 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:02.804904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1256/2488] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:02 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:02.808809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf700db-4194-4154-bddf-347a9c2e83f4 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:02.818879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf700db-4194-4154-bddf-347a9c2e83f4 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:02.875004 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373bcca-31fb-4a48-99ee-cf21f3404d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:02.875537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373bcca-31fb-4a48-99ee-cf21f3404d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:02.875846 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373bcca-31fb-4a48-99ee-cf21f3404d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:02.876195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373bcca-31fb-4a48-99ee-cf21f3404d21 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:02.876722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373bcca-31fb-4a48-99ee-cf21f3404d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:02.878692 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373bcca-31fb-4a48-99ee-cf21f3404d21 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:02.991362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1233/2489] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:02 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:02.995375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c576c2-38c9-47d8-a5cb-5b9cb65eab96 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:03.005676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c576c2-38c9-47d8-a5cb-5b9cb65eab96 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:03.010279 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b2e16-e7a4-4990-940b-1e7ea033a528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:03.010917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b2e16-e7a4-4990-940b-1e7ea033a528 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.011177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b2e16-e7a4-4990-940b-1e7ea033a528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.011447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b2e16-e7a4-4990-940b-1e7ea033a528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.012085 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b2e16-e7a4-4990-940b-1e7ea033a528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:03.013677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b2e16-e7a4-4990-940b-1e7ea033a528 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:03.023872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1257/2490] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:03.027726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373bcca-31fb-4a48-99ee-cf21f3404d21 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:03.038208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373bcca-31fb-4a48-99ee-cf21f3404d21 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:03.042891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975023b-cfad-4939-bf2a-8557cdaedc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:03.043487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975023b-cfad-4939-bf2a-8557cdaedc7d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.043767 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975023b-cfad-4939-bf2a-8557cdaedc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.044040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975023b-cfad-4939-bf2a-8557cdaedc7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.044674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975023b-cfad-4939-bf2a-8557cdaedc7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:03.046355 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975023b-cfad-4939-bf2a-8557cdaedc7d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:03.057040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1234/2491] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:03.060927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b2e16-e7a4-4990-940b-1e7ea033a528 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:03.071485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b2e16-e7a4-4990-940b-1e7ea033a528 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:03.076017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdfeb4-26f9-4fd8-bf83-9e9ad77f94ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:03.076650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdfeb4-26f9-4fd8-bf83-9e9ad77f94ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.076923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdfeb4-26f9-4fd8-bf83-9e9ad77f94ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.077191 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdfeb4-26f9-4fd8-bf83-9e9ad77f94ff admin admin] PATH_INFO: `/v3/projects/499dca24baed4a14a48db3e6a801046c/users/e1f6e73623594ae1bfdf822a5a92bd25/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.091788 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdfeb4-26f9-4fd8-bf83-9e9ad77f94ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=499dca24baed4a14a48db3e6a801046c, user_id=e1f6e73623594ae1bfdf822a5a92bd25, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:03.094434 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdfeb4-26f9-4fd8-bf83-9e9ad77f94ff admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:03.184645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1258/2492] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:03 2022] PUT /identity/v3/projects/499dca24baed4a14a48db3e6a801046c/users/e1f6e73623594ae1bfdf822a5a92bd25/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:03.197625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91664c8-7964-4044-a2dd-4df178a67fe2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.197912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91664c8-7964-4044-a2dd-4df178a67fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.198211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91664c8-7964-4044-a2dd-4df178a67fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.226974 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b91664c8-7964-4044-a2dd-4df178a67fe2 None None] MFA Rules not processed for user `e1f6e73623594ae1bfdf822a5a92bd25`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:03.228066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b91664c8-7964-4044-a2dd-4df178a67fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:03.234880 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:03.291384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1235/2493] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:03.295940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdfeb4-26f9-4fd8-bf83-9e9ad77f94ff admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:03.340557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:03.357002 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdfeb4-26f9-4fd8-bf83-9e9ad77f94ff admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:03.361623 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad6af8-2728-4721-9083-e2ef0328d304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:03.362244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad6af8-2728-4721-9083-e2ef0328d304 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.362527 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad6af8-2728-4721-9083-e2ef0328d304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.362826 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad6af8-2728-4721-9083-e2ef0328d304 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.363449 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad6af8-2728-4721-9083-e2ef0328d304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:03.365391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad6af8-2728-4721-9083-e2ef0328d304 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:03.457899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1259/2494] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:03 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:03.461993 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91664c8-7964-4044-a2dd-4df178a67fe2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:03.472697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91664c8-7964-4044-a2dd-4df178a67fe2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:03.532172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f9b57-37f7-4bec-9210-7861940e17e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:03.532798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f9b57-37f7-4bec-9210-7861940e17e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.533065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f9b57-37f7-4bec-9210-7861940e17e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.533338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f9b57-37f7-4bec-9210-7861940e17e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.533980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f9b57-37f7-4bec-9210-7861940e17e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:03.535982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f9b57-37f7-4bec-9210-7861940e17e1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:03.596169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1236/2495] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:03 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:03.600414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad6af8-2728-4721-9083-e2ef0328d304 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:03.611520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad6af8-2728-4721-9083-e2ef0328d304 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:03.616301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d9ca1-9988-4842-b596-e00d28a8030a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:03.616935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d9ca1-9988-4842-b596-e00d28a8030a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.617211 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d9ca1-9988-4842-b596-e00d28a8030a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.617482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d9ca1-9988-4842-b596-e00d28a8030a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.618115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d9ca1-9988-4842-b596-e00d28a8030a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:03.619761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d9ca1-9988-4842-b596-e00d28a8030a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:03.630106 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1260/2496] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:03.634164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f9b57-37f7-4bec-9210-7861940e17e1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:03.644097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f9b57-37f7-4bec-9210-7861940e17e1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:03.648653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccce4a-1cdd-495a-9981-3abfca21a6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:03.649300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccce4a-1cdd-495a-9981-3abfca21a6a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.649579 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccce4a-1cdd-495a-9981-3abfca21a6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.649850 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccce4a-1cdd-495a-9981-3abfca21a6a0 admin admin] PATH_INFO: `/v3/projects/727199bcb61943459ccbe3e1a75d4e19/users/81353f70202d437283215178ce308873/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.664032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccce4a-1cdd-495a-9981-3abfca21a6a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=727199bcb61943459ccbe3e1a75d4e19, user_id=81353f70202d437283215178ce308873, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:03.666718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccce4a-1cdd-495a-9981-3abfca21a6a0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:03.780147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1237/2497] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:03 2022] PUT /identity/v3/projects/727199bcb61943459ccbe3e1a75d4e19/users/81353f70202d437283215178ce308873/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:03.784674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d9ca1-9988-4842-b596-e00d28a8030a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:03.798370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d9ca1-9988-4842-b596-e00d28a8030a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:03.857649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afbdba-6acd-4fde-a856-f1d1e96a4367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:03.858277 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afbdba-6acd-4fde-a856-f1d1e96a4367 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.858552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afbdba-6acd-4fde-a856-f1d1e96a4367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.859491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afbdba-6acd-4fde-a856-f1d1e96a4367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.860231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afbdba-6acd-4fde-a856-f1d1e96a4367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:03.861802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afbdba-6acd-4fde-a856-f1d1e96a4367 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:03.871247 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1261/2498] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:03.876092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccce4a-1cdd-495a-9981-3abfca21a6a0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:03.886813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccce4a-1cdd-495a-9981-3abfca21a6a0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:03.896061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7a87d-b905-41d3-a595-bb0c2a0a8181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:03.896720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7a87d-b905-41d3-a595-bb0c2a0a8181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.896994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7a87d-b905-41d3-a595-bb0c2a0a8181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.897263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7a87d-b905-41d3-a595-bb0c2a0a8181 admin admin] PATH_INFO: `/v3/projects/727199bcb61943459ccbe3e1a75d4e19/users/81353f70202d437283215178ce308873/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.899200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7a87d-b905-41d3-a595-bb0c2a0a8181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=727199bcb61943459ccbe3e1a75d4e19, user_id=81353f70202d437283215178ce308873, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:03.901835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7a87d-b905-41d3-a595-bb0c2a0a8181 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:03.913061 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1238/2499] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:03 2022] PUT /identity/v3/projects/727199bcb61943459ccbe3e1a75d4e19/users/81353f70202d437283215178ce308873/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:03.924079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217775d-c5cc-41fa-803f-5954e97a3dc5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:03.924359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217775d-c5cc-41fa-803f-5954e97a3dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:03.924627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217775d-c5cc-41fa-803f-5954e97a3dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:03.952122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c217775d-c5cc-41fa-803f-5954e97a3dc5 None None] MFA Rules not processed for user `81353f70202d437283215178ce308873`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:03.953365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c217775d-c5cc-41fa-803f-5954e97a3dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:04.019229 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1262/2500] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:04.029916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7a87d-b905-41d3-a595-bb0c2a0a8181 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:04.032221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5fe7a87d-b905-41d3-a595-bb0c2a0a8181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:04.088659 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7a87d-b905-41d3-a595-bb0c2a0a8181 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:04.092357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020fc5c-6a4e-4edc-9019-d770e6d10cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:04.092952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020fc5c-6a4e-4edc-9019-d770e6d10cea service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:04.093737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020fc5c-6a4e-4edc-9019-d770e6d10cea service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:04.094003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020fc5c-6a4e-4edc-9019-d770e6d10cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:04.102984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020fc5c-6a4e-4edc-9019-d770e6d10cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:04.104410 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020fc5c-6a4e-4edc-9019-d770e6d10cea service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:04.114278 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1239/2501] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:04.587522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217775d-c5cc-41fa-803f-5954e97a3dc5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:04.597706 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217775d-c5cc-41fa-803f-5954e97a3dc5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:04.601723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc78d4b-6c16-4bf4-962d-df9fc37e3c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:04.602169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78d4b-6c16-4bf4-962d-df9fc37e3c5e service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:04.602464 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78d4b-6c16-4bf4-962d-df9fc37e3c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:04.602737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc78d4b-6c16-4bf4-962d-df9fc37e3c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:04.604841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2fc78d4b-6c16-4bf4-962d-df9fc37e3c5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:04.643258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:04.672000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78d4b-6c16-4bf4-962d-df9fc37e3c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:04.673556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc78d4b-6c16-4bf4-962d-df9fc37e3c5e service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:04.683295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1263/2502] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:04.777089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020fc5c-6a4e-4edc-9019-d770e6d10cea service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:04.781289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:04.840014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020fc5c-6a4e-4edc-9019-d770e6d10cea service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:04.844442 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:04.845066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:04.845345 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:04.845621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] PATH_INFO: `/v3/users/e1f6e73623594ae1bfdf822a5a92bd25` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:04.846825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1f6e73623594ae1bfdf822a5a92bd25)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:04.848714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:05.051280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1f6e73623594ae1bfdf822a5a92bd25'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:05.051942 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1f6e73623594ae1bfdf822a5a92bd25'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:05.064489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1f6e73623594ae1bfdf822a5a92bd25'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:05.116709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1f6e73623594ae1bfdf822a5a92bd25'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:05.117325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1f6e73623594ae1bfdf822a5a92bd25'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:05.130682 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1240/2503] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:04 2022] DELETE /identity/v3/users/e1f6e73623594ae1bfdf822a5a92bd25 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:05.135212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce48b-35ca-45bb-bd54-9caaec17b249 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:05.135489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce48b-35ca-45bb-bd54-9caaec17b249 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:05.135758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce48b-35ca-45bb-bd54-9caaec17b249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:05.153793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-bd7ce48b-35ca-45bb-bd54-9caaec17b249 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:05.155036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bd7ce48b-35ca-45bb-bd54-9caaec17b249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:05.213226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1264/2504] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:05.222869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:05.224271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:05.280408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49f81f-06b5-4229-8ba1-3dbb13cb5454 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:05.284611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cfade-39da-4b4e-9ef9-83db616f166d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:05.285231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cfade-39da-4b4e-9ef9-83db616f166d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:05.285511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cfade-39da-4b4e-9ef9-83db616f166d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:05.285785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cfade-39da-4b4e-9ef9-83db616f166d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:05.294678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cfade-39da-4b4e-9ef9-83db616f166d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:05.296086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cfade-39da-4b4e-9ef9-83db616f166d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:05.306025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1241/2505] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:05.713445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce48b-35ca-45bb-bd54-9caaec17b249 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:05.714919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bd7ce48b-35ca-45bb-bd54-9caaec17b249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:05.731267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce48b-35ca-45bb-bd54-9caaec17b249 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:05.736857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:05.737525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:05.738001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:05.738254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] PATH_INFO: `/v3/projects/499dca24baed4a14a48db3e6a801046c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:05.739739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=499dca24baed4a14a48db3e6a801046c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:05.741651 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:05.871264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '499dca24baed4a14a48db3e6a801046c', 'request_id': 'req-aaacd60d-afe1-4ce5-833d-0f6a259e3295'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:05.877506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '499dca24baed4a14a48db3e6a801046c', 'request_id': 'req-aaacd60d-afe1-4ce5-833d-0f6a259e3295'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:05.957390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '499dca24baed4a14a48db3e6a801046c', 'request_id': 'req-aaacd60d-afe1-4ce5-833d-0f6a259e3295'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:05.969219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] The token cache is being invalidate because project 499dca24baed4a14a48db3e6a801046c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:05.969505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:05.970114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:05.971654 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1265/2506] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:05 2022] DELETE /identity/v3/projects/499dca24baed4a14a48db3e6a801046c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:05.975728 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cfade-39da-4b4e-9ef9-83db616f166d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:05.985116 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cfade-39da-4b4e-9ef9-83db616f166d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:06.007543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:06.049291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:06.049924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:06.050234 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:06.050511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] PATH_INFO: `/v3/users/81353f70202d437283215178ce308873` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:06.051666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81353f70202d437283215178ce308873)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:06.053567 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:06.280741 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81353f70202d437283215178ce308873'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:06.281393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81353f70202d437283215178ce308873'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:06.293284 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81353f70202d437283215178ce308873'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:06.368563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81353f70202d437283215178ce308873'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:06.369182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81353f70202d437283215178ce308873'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:06.382510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1242/2507] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:05 2022] DELETE /identity/v3/users/81353f70202d437283215178ce308873 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:06.782883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:06.785103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:06.811480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacd60d-afe1-4ce5-833d-0f6a259e3295 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:06.862505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:06.869679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:06.870476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:06.870784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:06.871055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] PATH_INFO: `/v3/projects/727199bcb61943459ccbe3e1a75d4e19` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:06.872175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=727199bcb61943459ccbe3e1a75d4e19)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:06.874221 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:06.973716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '727199bcb61943459ccbe3e1a75d4e19', 'request_id': 'req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:06.980802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '727199bcb61943459ccbe3e1a75d4e19', 'request_id': 'req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:06.999760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:06.999760 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:07.012456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec5e9-7da1-46f6-9c73-dcabe60c9432 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:07.065492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '727199bcb61943459ccbe3e1a75d4e19', 'request_id': 'req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:07.069618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:07.070273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.070561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.070839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] PATH_INFO: `/v3/users/dc6572065fca458299c5485f5ae28661` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.071956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6572065fca458299c5485f5ae28661)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:07.073889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:07.076927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] The token cache is being invalidate because project 727199bcb61943459ccbe3e1a75d4e19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:07.077217 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:07.078555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-39fe99ae-42dc-4ee8-88fb-373d0d38c5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:07.080084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1266/2508] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:06 2022] DELETE /identity/v3/projects/727199bcb61943459ccbe3e1a75d4e19 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:07.084809 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbff6c-d41f-48c8-ae5e-dc0b95c4d9fa None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.085579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbff6c-d41f-48c8-ae5e-dc0b95c4d9fa None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.085579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbff6c-d41f-48c8-ae5e-dc0b95c4d9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.105881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-3bbbff6c-d41f-48c8-ae5e-dc0b95c4d9fa None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:07.107153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3bbbff6c-d41f-48c8-ae5e-dc0b95c4d9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:07.115887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1267/2509] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:07.123524 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd759c9-3533-4064-9147-6d27cf5c7792 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.123802 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd759c9-3533-4064-9147-6d27cf5c7792 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.124101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd759c9-3533-4064-9147-6d27cf5c7792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.142637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2cd759c9-3533-4064-9147-6d27cf5c7792 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:07.143795 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2cd759c9-3533-4064-9147-6d27cf5c7792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:07.151365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1268/2510] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:07.155189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd759c9-3533-4064-9147-6d27cf5c7792 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:07.164055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd759c9-3533-4064-9147-6d27cf5c7792 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:07.168507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40af090-634d-4887-8ded-2bc0e7771936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:07.169065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40af090-634d-4887-8ded-2bc0e7771936 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.169342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40af090-634d-4887-8ded-2bc0e7771936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.169608 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40af090-634d-4887-8ded-2bc0e7771936 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.170261 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40af090-634d-4887-8ded-2bc0e7771936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:50:07.170563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40af090-634d-4887-8ded-2bc0e7771936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:07.172911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40af090-634d-4887-8ded-2bc0e7771936 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:07.184933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1269/2511] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:50:07.188973 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40af090-634d-4887-8ded-2bc0e7771936 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:07.198873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40af090-634d-4887-8ded-2bc0e7771936 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:07.203857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a08cc-0f98-40db-b202-eef1c702c244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:07.204548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a08cc-0f98-40db-b202-eef1c702c244 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.204824 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a08cc-0f98-40db-b202-eef1c702c244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.205093 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a08cc-0f98-40db-b202-eef1c702c244 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.205695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a08cc-0f98-40db-b202-eef1c702c244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:07.207577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a08cc-0f98-40db-b202-eef1c702c244 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:07.261427 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:07.263096 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6572065fca458299c5485f5ae28661'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:07.263749 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6572065fca458299c5485f5ae28661'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:07.275980 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6572065fca458299c5485f5ae28661'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:07.312296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1270/2512] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:07 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:07.317175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a08cc-0f98-40db-b202-eef1c702c244 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:07.318703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aa0a08cc-0f98-40db-b202-eef1c702c244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:07.352216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6572065fca458299c5485f5ae28661'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:07.352942 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7e652900-7b39-4fe1-a6eb-c749144aaa4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6572065fca458299c5485f5ae28661'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:07.365795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1243/2513] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:06 2022] DELETE /identity/v3/users/dc6572065fca458299c5485f5ae28661 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:07.370083 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f03aab-9b29-494d-b890-ae766f5da92c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.370398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f03aab-9b29-494d-b890-ae766f5da92c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.370697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f03aab-9b29-494d-b890-ae766f5da92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.381847 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a08cc-0f98-40db-b202-eef1c702c244 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:07.389306 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f4f03aab-9b29-494d-b890-ae766f5da92c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:07.391025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4f03aab-9b29-494d-b890-ae766f5da92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:07.423558 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:07.438389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d20f8-afcd-44c5-a941-0405586ed70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:07.438993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d20f8-afcd-44c5-a941-0405586ed70e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.439267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d20f8-afcd-44c5-a941-0405586ed70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.442564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d20f8-afcd-44c5-a941-0405586ed70e admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.442564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d20f8-afcd-44c5-a941-0405586ed70e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:07.442564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d20f8-afcd-44c5-a941-0405586ed70e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:07.450613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1244/2514] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:07.460807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f03aab-9b29-494d-b890-ae766f5da92c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:07.462127 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f4f03aab-9b29-494d-b890-ae766f5da92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:07.500206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1271/2515] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:07 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:07.504254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d20f8-afcd-44c5-a941-0405586ed70e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:07.514179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d20f8-afcd-44c5-a941-0405586ed70e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:07.520100 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dafd3-c160-4e8e-85ca-a41c3228aeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:07.520781 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dafd3-c160-4e8e-85ca-a41c3228aeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.521065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dafd3-c160-4e8e-85ca-a41c3228aeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.521178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f03aab-9b29-494d-b890-ae766f5da92c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:07.521355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5dafd3-c160-4e8e-85ca-a41c3228aeb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.522166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5dafd3-c160-4e8e-85ca-a41c3228aeb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:07.523778 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5dafd3-c160-4e8e-85ca-a41c3228aeb7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:07.524854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14057e0b-c828-4119-a075-538ce686200d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:07.525530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14057e0b-c828-4119-a075-538ce686200d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.525797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14057e0b-c828-4119-a075-538ce686200d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.526066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14057e0b-c828-4119-a075-538ce686200d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.535338 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1272/2516] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:07.538808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14057e0b-c828-4119-a075-538ce686200d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:07.542108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dafd3-c160-4e8e-85ca-a41c3228aeb7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:07.542243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14057e0b-c828-4119-a075-538ce686200d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:07.549251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5dafd3-c160-4e8e-85ca-a41c3228aeb7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:07.549828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1245/2517] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:50:07.554041 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea2fc8-f000-4db6-966d-a9b52da9d294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:07.554842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea2fc8-f000-4db6-966d-a9b52da9d294 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.555110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea2fc8-f000-4db6-966d-a9b52da9d294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.555596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea2fc8-f000-4db6-966d-a9b52da9d294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.556346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea2fc8-f000-4db6-966d-a9b52da9d294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:07.558374 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea2fc8-f000-4db6-966d-a9b52da9d294 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:07.574080 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1273/2518] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:07.574591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14057e0b-c828-4119-a075-538ce686200d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:07.585024 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14057e0b-c828-4119-a075-538ce686200d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:07.595992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b608678-84c8-41fd-be51-a6a9728b84bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:07.596723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b608678-84c8-41fd-be51-a6a9728b84bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.597002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b608678-84c8-41fd-be51-a6a9728b84bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.597277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b608678-84c8-41fd-be51-a6a9728b84bf admin admin] PATH_INFO: `/v3/projects/8207d8781edd40f3b4087a601696d5f3/users/5e4a79b8857f40b8bd9e28e390148066/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.612625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b608678-84c8-41fd-be51-a6a9728b84bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8207d8781edd40f3b4087a601696d5f3, user_id=5e4a79b8857f40b8bd9e28e390148066, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:07.615319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b608678-84c8-41fd-be51-a6a9728b84bf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:07.682986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1246/2519] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:07 2022] PUT /identity/v3/projects/8207d8781edd40f3b4087a601696d5f3/users/5e4a79b8857f40b8bd9e28e390148066/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:07.688471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65361427-cc41-450f-a4ba-491735faf18c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.688751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65361427-cc41-450f-a4ba-491735faf18c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.689019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65361427-cc41-450f-a4ba-491735faf18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.709090 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-65361427-cc41-450f-a4ba-491735faf18c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:07.710743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-65361427-cc41-450f-a4ba-491735faf18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:07.788246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1274/2520] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:07.804496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b608678-84c8-41fd-be51-a6a9728b84bf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:07.806095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8b608678-84c8-41fd-be51-a6a9728b84bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:07.859218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:07.870944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b608678-84c8-41fd-be51-a6a9728b84bf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:07.875110 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7db71d-14ec-48cf-8b90-2004c9b3c9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:07.875730 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db71d-14ec-48cf-8b90-2004c9b3c9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:07.876006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db71d-14ec-48cf-8b90-2004c9b3c9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:07.876274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db71d-14ec-48cf-8b90-2004c9b3c9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:07.885111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7db71d-14ec-48cf-8b90-2004c9b3c9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:07.886574 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7db71d-14ec-48cf-8b90-2004c9b3c9a2 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:07.896938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1247/2521] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:07.976406 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65361427-cc41-450f-a4ba-491735faf18c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:07.977858 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-65361427-cc41-450f-a4ba-491735faf18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:08.001548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65361427-cc41-450f-a4ba-491735faf18c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:08.014130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:08.014855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:08.015143 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:08.015417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] PATH_INFO: `/v3/projects/7b43ccfbf77d42dcbab0b85aceeb4324` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:08.017678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b43ccfbf77d42dcbab0b85aceeb4324)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:08.018941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:08.123419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b43ccfbf77d42dcbab0b85aceeb4324', 'request_id': 'req-d51cdb56-7b95-403e-9941-62e2dc21fc37'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:08.130478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b43ccfbf77d42dcbab0b85aceeb4324', 'request_id': 'req-d51cdb56-7b95-403e-9941-62e2dc21fc37'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:08.203118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b43ccfbf77d42dcbab0b85aceeb4324', 'request_id': 'req-d51cdb56-7b95-403e-9941-62e2dc21fc37'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:08.220302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] The token cache is being invalidate because project 7b43ccfbf77d42dcbab0b85aceeb4324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:08.220643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:08.221353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d51cdb56-7b95-403e-9941-62e2dc21fc37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:08.223872 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1275/2522] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:07 2022] DELETE /identity/v3/projects/7b43ccfbf77d42dcbab0b85aceeb4324 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:08.229021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a11d09-8353-4c4b-af3b-369cd7fb5787 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:08.229862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a11d09-8353-4c4b-af3b-369cd7fb5787 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:08.230478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a11d09-8353-4c4b-af3b-369cd7fb5787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:08.256247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-62a11d09-8353-4c4b-af3b-369cd7fb5787 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:08.257516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-62a11d09-8353-4c4b-af3b-369cd7fb5787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:08.318969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1248/2523] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:08.326680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee780f-8108-484b-a5d9-443f9faf5c75 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:08.326965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee780f-8108-484b-a5d9-443f9faf5c75 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:08.327242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee780f-8108-484b-a5d9-443f9faf5c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:08.346927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b6ee780f-8108-484b-a5d9-443f9faf5c75 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:08.348123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b6ee780f-8108-484b-a5d9-443f9faf5c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:08.355967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1276/2524] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:08.360466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a11d09-8353-4c4b-af3b-369cd7fb5787 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:08.370040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a11d09-8353-4c4b-af3b-369cd7fb5787 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:08.374725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74160c-da66-4fcd-b28f-9faece2136ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:08.375319 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74160c-da66-4fcd-b28f-9faece2136ce admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:08.375608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74160c-da66-4fcd-b28f-9faece2136ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:08.375879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74160c-da66-4fcd-b28f-9faece2136ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:08.376521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74160c-da66-4fcd-b28f-9faece2136ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:50:08.376825 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74160c-da66-4fcd-b28f-9faece2136ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:08.378457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74160c-da66-4fcd-b28f-9faece2136ce admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:08.389843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1249/2525] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:50:08.393968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee780f-8108-484b-a5d9-443f9faf5c75 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:08.403107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee780f-8108-484b-a5d9-443f9faf5c75 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:08.408290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fa4cd-1583-454b-ac44-96e1f358d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:08.408878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fa4cd-1583-454b-ac44-96e1f358d7ca admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:08.409157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fa4cd-1583-454b-ac44-96e1f358d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:08.410511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fa4cd-1583-454b-ac44-96e1f358d7ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:08.410511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51fa4cd-1583-454b-ac44-96e1f358d7ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:08.411959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51fa4cd-1583-454b-ac44-96e1f358d7ca admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:08.555913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1277/2526] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:08 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:08.557230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74160c-da66-4fcd-b28f-9faece2136ce admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:08.568300 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74160c-da66-4fcd-b28f-9faece2136ce admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:08.629401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322fdcf-7182-4ceb-91ae-87fcf870225b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:08.630010 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322fdcf-7182-4ceb-91ae-87fcf870225b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:08.630374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322fdcf-7182-4ceb-91ae-87fcf870225b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:08.630645 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322fdcf-7182-4ceb-91ae-87fcf870225b admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:08.631266 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322fdcf-7182-4ceb-91ae-87fcf870225b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:08.633152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322fdcf-7182-4ceb-91ae-87fcf870225b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:08.758207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1250/2527] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:08 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:08.762454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fa4cd-1583-454b-ac44-96e1f358d7ca admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:08.773378 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fa4cd-1583-454b-ac44-96e1f358d7ca admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:08.778013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67125af1-556a-41e8-a235-d17b86409db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:08.784193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67125af1-556a-41e8-a235-d17b86409db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:08.784193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67125af1-556a-41e8-a235-d17b86409db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:08.784193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67125af1-556a-41e8-a235-d17b86409db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:08.784193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67125af1-556a-41e8-a235-d17b86409db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:08.784193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67125af1-556a-41e8-a235-d17b86409db1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:08.784193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:08.792976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1278/2528] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:08.797505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322fdcf-7182-4ceb-91ae-87fcf870225b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:08.807937 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322fdcf-7182-4ceb-91ae-87fcf870225b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:08.813230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc350000-9677-4edd-b4a7-2cf89ac6a08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:08.813881 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc350000-9677-4edd-b4a7-2cf89ac6a08b admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:08.814243 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc350000-9677-4edd-b4a7-2cf89ac6a08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:08.814518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc350000-9677-4edd-b4a7-2cf89ac6a08b admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:08.815155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc350000-9677-4edd-b4a7-2cf89ac6a08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:08.817089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc350000-9677-4edd-b4a7-2cf89ac6a08b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:08.827779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1251/2529] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:08.834516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67125af1-556a-41e8-a235-d17b86409db1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:08.844231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67125af1-556a-41e8-a235-d17b86409db1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:08.849915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af671a8-e53b-4308-93cc-1116cd7e0075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:08.850659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af671a8-e53b-4308-93cc-1116cd7e0075 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:08.850937 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af671a8-e53b-4308-93cc-1116cd7e0075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:08.851215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af671a8-e53b-4308-93cc-1116cd7e0075 admin admin] PATH_INFO: `/v3/projects/1d054fe5fc3648de8cb9be51aaec187c/users/e1ad71c0bab546788d2d2468165d1b04/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:08.866436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af671a8-e53b-4308-93cc-1116cd7e0075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d054fe5fc3648de8cb9be51aaec187c, user_id=e1ad71c0bab546788d2d2468165d1b04, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:08.869138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af671a8-e53b-4308-93cc-1116cd7e0075 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:08.941502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1279/2530] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:08 2022] PUT /identity/v3/projects/1d054fe5fc3648de8cb9be51aaec187c/users/e1ad71c0bab546788d2d2468165d1b04/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:08.953272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b2e42-2c9e-4ed8-ab5a-d45b1e178111 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:08.953603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b2e42-2c9e-4ed8-ab5a-d45b1e178111 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:08.953887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b2e42-2c9e-4ed8-ab5a-d45b1e178111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:08.983238 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-e10b2e42-2c9e-4ed8-ab5a-d45b1e178111 None None] MFA Rules not processed for user `e1ad71c0bab546788d2d2468165d1b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:08.985087 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e10b2e42-2c9e-4ed8-ab5a-d45b1e178111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:09.029302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:09.055732 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1252/2531] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:09.060531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af671a8-e53b-4308-93cc-1116cd7e0075 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.123620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af671a8-e53b-4308-93cc-1116cd7e0075 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:09.128318 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa1f7e-5e80-409a-a908-0f4f5511b918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:09.128920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa1f7e-5e80-409a-a908-0f4f5511b918 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.129198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa1f7e-5e80-409a-a908-0f4f5511b918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.129471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa1f7e-5e80-409a-a908-0f4f5511b918 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.130177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa1f7e-5e80-409a-a908-0f4f5511b918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:09.132091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa1f7e-5e80-409a-a908-0f4f5511b918 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:09.226166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1280/2532] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:09 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:09.227315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b2e42-2c9e-4ed8-ab5a-d45b1e178111 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.238666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b2e42-2c9e-4ed8-ab5a-d45b1e178111 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:09.273216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236da2aa-0f7b-4e0b-ae70-0f60e26d8e87 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.273501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236da2aa-0f7b-4e0b-ae70-0f60e26d8e87 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.273772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236da2aa-0f7b-4e0b-ae70-0f60e26d8e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.305231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-236da2aa-0f7b-4e0b-ae70-0f60e26d8e87 None None] MFA Rules not processed for user `5e4a79b8857f40b8bd9e28e390148066`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:09.305440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033f643-9a61-45bc-82b0-2cd56460874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:09.306039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033f643-9a61-45bc-82b0-2cd56460874a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.306357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033f643-9a61-45bc-82b0-2cd56460874a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.306528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-236da2aa-0f7b-4e0b-ae70-0f60e26d8e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:09.306656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033f643-9a61-45bc-82b0-2cd56460874a admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.307273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033f643-9a61-45bc-82b0-2cd56460874a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:09.309245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033f643-9a61-45bc-82b0-2cd56460874a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:09.376305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1281/2533] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:09.380361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236da2aa-0f7b-4e0b-ae70-0f60e26d8e87 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.381741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-236da2aa-0f7b-4e0b-ae70-0f60e26d8e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:09.395490 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236da2aa-0f7b-4e0b-ae70-0f60e26d8e87 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:09.402967 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1253/2534] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:09 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:09.403092 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d25221-852a-424b-8887-8d968502a4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:09.403353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d25221-852a-424b-8887-8d968502a4d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.403629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d25221-852a-424b-8887-8d968502a4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.403901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d25221-852a-424b-8887-8d968502a4d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.404018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033f643-9a61-45bc-82b0-2cd56460874a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.404531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d25221-852a-424b-8887-8d968502a4d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:09.407089 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d25221-852a-424b-8887-8d968502a4d4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:09.414426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033f643-9a61-45bc-82b0-2cd56460874a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:09.419121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aec404-6fcb-43c9-8884-9aeaed3608e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:09.424301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aec404-6fcb-43c9-8884-9aeaed3608e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.424301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aec404-6fcb-43c9-8884-9aeaed3608e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.424301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aec404-6fcb-43c9-8884-9aeaed3608e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.424301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aec404-6fcb-43c9-8884-9aeaed3608e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:09.424301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aec404-6fcb-43c9-8884-9aeaed3608e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:09.433562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1254/2535] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:09.437556 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aec404-6fcb-43c9-8884-9aeaed3608e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.453039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aec404-6fcb-43c9-8884-9aeaed3608e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:09.453039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d805eb-1113-4611-b377-82f443bd295c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:09.453039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d805eb-1113-4611-b377-82f443bd295c admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.453039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d805eb-1113-4611-b377-82f443bd295c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.453467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d805eb-1113-4611-b377-82f443bd295c admin admin] PATH_INFO: `/v3/projects/636860aaeaa14048883741791d4c92e1/users/c8d23412b87f4f4ca037b3866be14653/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.467982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d805eb-1113-4611-b377-82f443bd295c admin admin] RBAC: Authorizing `identity:create_grant(project_id=636860aaeaa14048883741791d4c92e1, user_id=c8d23412b87f4f4ca037b3866be14653, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:09.471318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d805eb-1113-4611-b377-82f443bd295c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:09.515442 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1282/2536] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:09 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:09.516526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d25221-852a-424b-8887-8d968502a4d4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.536054 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1255/2537] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:09 2022] PUT /identity/v3/projects/636860aaeaa14048883741791d4c92e1/users/c8d23412b87f4f4ca037b3866be14653/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:09.539465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d805eb-1113-4611-b377-82f443bd295c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.546793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:09.584879 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d25221-852a-424b-8887-8d968502a4d4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:09.612036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d805eb-1113-4611-b377-82f443bd295c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:09.617175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2032bc-35d0-4356-943f-78b709fd5a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:09.617778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2032bc-35d0-4356-943f-78b709fd5a65 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.618185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2032bc-35d0-4356-943f-78b709fd5a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.618418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2032bc-35d0-4356-943f-78b709fd5a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.619068 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2032bc-35d0-4356-943f-78b709fd5a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:09.620788 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2032bc-35d0-4356-943f-78b709fd5a65 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:09.630965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1256/2538] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:09.631790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2032bc-35d0-4356-943f-78b709fd5a65 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.635377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-be2032bc-35d0-4356-943f-78b709fd5a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:09.646258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c21a90-4a6e-4344-9041-a0e7f9d02c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:09.646948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c21a90-4a6e-4344-9041-a0e7f9d02c27 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.647227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c21a90-4a6e-4344-9041-a0e7f9d02c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.647496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c21a90-4a6e-4344-9041-a0e7f9d02c27 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.648119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c21a90-4a6e-4344-9041-a0e7f9d02c27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:09.649558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2032bc-35d0-4356-943f-78b709fd5a65 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:09.650021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c21a90-4a6e-4344-9041-a0e7f9d02c27 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:09.654404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:09.655015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.655293 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.655628 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] PATH_INFO: `/v3/users/8b6f0b6242884bde96d3cb0f9719ffcc` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.657233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6f0b6242884bde96d3cb0f9719ffcc)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:09.659153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:09.773618 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1283/2539] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:09 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:09.774545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c21a90-4a6e-4344-9041-a0e7f9d02c27 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.781546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:09.785298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c21a90-4a6e-4344-9041-a0e7f9d02c27 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:09.790016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51f2f5-a9ee-47cc-ac9f-35542cc0d066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:09.790693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51f2f5-a9ee-47cc-ac9f-35542cc0d066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.790966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51f2f5-a9ee-47cc-ac9f-35542cc0d066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.791235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51f2f5-a9ee-47cc-ac9f-35542cc0d066 admin admin] PATH_INFO: `/v3/projects/636860aaeaa14048883741791d4c92e1/users/c8d23412b87f4f4ca037b3866be14653/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.793140 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51f2f5-a9ee-47cc-ac9f-35542cc0d066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636860aaeaa14048883741791d4c92e1, user_id=c8d23412b87f4f4ca037b3866be14653, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:09.795912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51f2f5-a9ee-47cc-ac9f-35542cc0d066 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:09.806130 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1284/2540] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:09 2022] PUT /identity/v3/projects/636860aaeaa14048883741791d4c92e1/users/c8d23412b87f4f4ca037b3866be14653/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:09.807122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51f2f5-a9ee-47cc-ac9f-35542cc0d066 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.869537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51f2f5-a9ee-47cc-ac9f-35542cc0d066 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:09.874153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8afcd7-20d7-4b01-af63-6d23795da520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:09.875275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8afcd7-20d7-4b01-af63-6d23795da520 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.875551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8afcd7-20d7-4b01-af63-6d23795da520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.875821 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8afcd7-20d7-4b01-af63-6d23795da520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.876478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8afcd7-20d7-4b01-af63-6d23795da520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:09.878074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8afcd7-20d7-4b01-af63-6d23795da520 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:09.887517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1285/2541] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:09.891886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e41532-ed99-424e-b5e5-23391439632f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:09.891886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e41532-ed99-424e-b5e5-23391439632f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:09.891886 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e41532-ed99-424e-b5e5-23391439632f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:09.917190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b9e41532-ed99-424e-b5e5-23391439632f None None] MFA Rules not processed for user `c8d23412b87f4f4ca037b3866be14653`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:09.918583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b9e41532-ed99-424e-b5e5-23391439632f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:09.925649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6f0b6242884bde96d3cb0f9719ffcc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:09.926363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6f0b6242884bde96d3cb0f9719ffcc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:09.939810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6f0b6242884bde96d3cb0f9719ffcc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:09.951341 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:09.986114 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1286/2542] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:09.987018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e41532-ed99-424e-b5e5-23391439632f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:09.988495 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b9e41532-ed99-424e-b5e5-23391439632f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:10.001906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e41532-ed99-424e-b5e5-23391439632f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:10.006508 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4a730-c87a-4897-b70b-491ee5d27f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:10.007193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4a730-c87a-4897-b70b-491ee5d27f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:10.007473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4a730-c87a-4897-b70b-491ee5d27f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:10.007747 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4a730-c87a-4897-b70b-491ee5d27f2e admin admin] PATH_INFO: `/v3/projects/416c5642950a48d9a2faeadc45f6e8a4/users/f4480b1b068c48cb93f789dd22a6766b/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:10.009139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6f0b6242884bde96d3cb0f9719ffcc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:10.010302 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6f0b6242884bde96d3cb0f9719ffcc'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:10.023828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4a730-c87a-4897-b70b-491ee5d27f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=416c5642950a48d9a2faeadc45f6e8a4, user_id=f4480b1b068c48cb93f789dd22a6766b, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:10.026547 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4a730-c87a-4897-b70b-491ee5d27f2e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:10.027523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1257/2543] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:09 2022] DELETE /identity/v3/users/8b6f0b6242884bde96d3cb0f9719ffcc => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:10.028414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:10.029848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:10.085845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c134f-7b3f-4a11-895a-d2b312463498 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:10.089445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec006f9-e406-4c57-bd41-0b5a72e5ed31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:10.090032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec006f9-e406-4c57-bd41-0b5a72e5ed31 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:10.090335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec006f9-e406-4c57-bd41-0b5a72e5ed31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:10.090605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec006f9-e406-4c57-bd41-0b5a72e5ed31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:10.092395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bec006f9-e406-4c57-bd41-0b5a72e5ed31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:10.117401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1287/2544] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:09 2022] PUT /identity/v3/projects/416c5642950a48d9a2faeadc45f6e8a4/users/f4480b1b068c48cb93f789dd22a6766b/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:10.118900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1785ad4-5f83-43e0-ae8d-105da8374e0f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:10.119182 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1785ad4-5f83-43e0-ae8d-105da8374e0f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:10.119486 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1785ad4-5f83-43e0-ae8d-105da8374e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:10.138546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e1785ad4-5f83-43e0-ae8d-105da8374e0f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:10.139859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e1785ad4-5f83-43e0-ae8d-105da8374e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:10.153713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec006f9-e406-4c57-bd41-0b5a72e5ed31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:10.155164 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec006f9-e406-4c57-bd41-0b5a72e5ed31 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:10.164646 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1258/2545] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:10.165642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec006f9-e406-4c57-bd41-0b5a72e5ed31 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:10.205733 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1288/2546] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:10.215383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1785ad4-5f83-43e0-ae8d-105da8374e0f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:10.216746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e1785ad4-5f83-43e0-ae8d-105da8374e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:10.234726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:10.238816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec006f9-e406-4c57-bd41-0b5a72e5ed31 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:10.246747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362069d-dc09-48e4-8bee-75e1c45ae430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:10.247678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362069d-dc09-48e4-8bee-75e1c45ae430 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:10.247828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362069d-dc09-48e4-8bee-75e1c45ae430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:10.248058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362069d-dc09-48e4-8bee-75e1c45ae430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:10.248698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362069d-dc09-48e4-8bee-75e1c45ae430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:10.250829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362069d-dc09-48e4-8bee-75e1c45ae430 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:10.261481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1259/2547] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:10.266885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362069d-dc09-48e4-8bee-75e1c45ae430 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:10.277620 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362069d-dc09-48e4-8bee-75e1c45ae430 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:10.280415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1785ad4-5f83-43e0-ae8d-105da8374e0f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:10.283160 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b549d-9a96-4954-b1a4-5e3226200482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:10.283806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b549d-9a96-4954-b1a4-5e3226200482 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:10.284086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b549d-9a96-4954-b1a4-5e3226200482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:10.284572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b549d-9a96-4954-b1a4-5e3226200482 admin admin] PATH_INFO: `/v3/projects/416c5642950a48d9a2faeadc45f6e8a4/users/f4480b1b068c48cb93f789dd22a6766b/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:10.284923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4acdd-1e88-4ac1-90cb-5b4cac7012ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:10.285525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4acdd-1e88-4ac1-90cb-5b4cac7012ab service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:10.285807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4acdd-1e88-4ac1-90cb-5b4cac7012ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:10.286081 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4acdd-1e88-4ac1-90cb-5b4cac7012ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:10.286892 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b549d-9a96-4954-b1a4-5e3226200482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=416c5642950a48d9a2faeadc45f6e8a4, user_id=f4480b1b068c48cb93f789dd22a6766b, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:10.289688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b549d-9a96-4954-b1a4-5e3226200482 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:10.295682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4acdd-1e88-4ac1-90cb-5b4cac7012ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:10.297148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4acdd-1e88-4ac1-90cb-5b4cac7012ab service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:10.301848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1260/2548] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:10 2022] PUT /identity/v3/projects/416c5642950a48d9a2faeadc45f6e8a4/users/f4480b1b068c48cb93f789dd22a6766b/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:10.311363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1289/2549] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:50:10.771922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b549d-9a96-4954-b1a4-5e3226200482 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:10.775154 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-701b549d-9a96-4954-b1a4-5e3226200482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:10.872964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b549d-9a96-4954-b1a4-5e3226200482 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:10.878909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:10.879550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:10.879832 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:10.880107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] PATH_INFO: `/v3/projects/f7248edf24e34094ab2dfd803073241d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:10.881232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7248edf24e34094ab2dfd803073241d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:10.883190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:11.009963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7248edf24e34094ab2dfd803073241d', 'request_id': 'req-7a4cadab-2330-4067-a72d-d08ce4e858dd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:11.016621 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7248edf24e34094ab2dfd803073241d', 'request_id': 'req-7a4cadab-2330-4067-a72d-d08ce4e858dd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:11.088919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7248edf24e34094ab2dfd803073241d', 'request_id': 'req-7a4cadab-2330-4067-a72d-d08ce4e858dd'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:11.100273 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] The token cache is being invalidate because project f7248edf24e34094ab2dfd803073241d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:11.100588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:11.101244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-7a4cadab-2330-4067-a72d-d08ce4e858dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:11.102772 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1261/2550] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:10 2022] DELETE /identity/v3/projects/f7248edf24e34094ab2dfd803073241d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:11.106768 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4acdd-1e88-4ac1-90cb-5b4cac7012ab service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:11.116807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4acdd-1e88-4ac1-90cb-5b4cac7012ab service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:11.161512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:11.173774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:11.174439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:11.174722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:11.174995 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] PATH_INFO: `/v3/users/8a18e15ca9af46b3abb6ba972320bb65` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:11.176111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a18e15ca9af46b3abb6ba972320bb65)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:11.178024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:11.370417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a18e15ca9af46b3abb6ba972320bb65'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:11.382355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a18e15ca9af46b3abb6ba972320bb65'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:11.383021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a18e15ca9af46b3abb6ba972320bb65'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:11.482958 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a18e15ca9af46b3abb6ba972320bb65'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:11.483689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a18e15ca9af46b3abb6ba972320bb65'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:11.497373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1290/2551] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:11 2022] DELETE /identity/v3/users/8a18e15ca9af46b3abb6ba972320bb65 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:11.527935 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177046f4-493c-48b3-adc6-0c9e1ce1c040 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:11.528439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177046f4-493c-48b3-adc6-0c9e1ce1c040 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:11.528537 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177046f4-493c-48b3-adc6-0c9e1ce1c040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:11.558830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-177046f4-493c-48b3-adc6-0c9e1ce1c040 None None] MFA Rules not processed for user `f4480b1b068c48cb93f789dd22a6766b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:11.560052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-177046f4-493c-48b3-adc6-0c9e1ce1c040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:11.626759 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:11.635715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1262/2552] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:11.647545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:11.648998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:11.707804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7c64b-fb45-4173-864b-c4773ce03867 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:11.711636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703dc0b-a6f8-4b9e-88cd-99f14f0e6acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:11.712265 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703dc0b-a6f8-4b9e-88cd-99f14f0e6acc service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:11.712596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703dc0b-a6f8-4b9e-88cd-99f14f0e6acc service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:11.712870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703dc0b-a6f8-4b9e-88cd-99f14f0e6acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:11.714617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0703dc0b-a6f8-4b9e-88cd-99f14f0e6acc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:11.793007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703dc0b-a6f8-4b9e-88cd-99f14f0e6acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:11.794472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703dc0b-a6f8-4b9e-88cd-99f14f0e6acc service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:11.803946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1291/2553] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:11.976632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177046f4-493c-48b3-adc6-0c9e1ce1c040 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:11.978490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-177046f4-493c-48b3-adc6-0c9e1ce1c040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:12.057857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177046f4-493c-48b3-adc6-0c9e1ce1c040 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:12.064981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:12.065618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:12.065899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:12.066474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] PATH_INFO: `/v3/projects/dfd3423a51df420aa5e9323f8bafba08` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:12.067617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd3423a51df420aa5e9323f8bafba08)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:12.069516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:12.192317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd3423a51df420aa5e9323f8bafba08', 'request_id': 'req-a1b206f0-644f-4d97-9033-af4f80c8a558'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:12.198585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd3423a51df420aa5e9323f8bafba08', 'request_id': 'req-a1b206f0-644f-4d97-9033-af4f80c8a558'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:12.288667 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd3423a51df420aa5e9323f8bafba08', 'request_id': 'req-a1b206f0-644f-4d97-9033-af4f80c8a558'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:12.301150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] The token cache is being invalidate because project dfd3423a51df420aa5e9323f8bafba08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:12.301647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:12.302391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:12.303988 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1263/2554] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:11 2022] DELETE /identity/v3/projects/dfd3423a51df420aa5e9323f8bafba08 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:12.308061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703dc0b-a6f8-4b9e-88cd-99f14f0e6acc service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:12.320451 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703dc0b-a6f8-4b9e-88cd-99f14f0e6acc service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:12.339086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:12.376990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:12.377643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:12.377923 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:12.378372 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] PATH_INFO: `/v3/users/0c171dee265246cd8554c20f10025052` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:12.379497 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c171dee265246cd8554c20f10025052)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:12.381414 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:12.565699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c171dee265246cd8554c20f10025052'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:12.578028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c171dee265246cd8554c20f10025052'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:12.578727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c171dee265246cd8554c20f10025052'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:12.642998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c171dee265246cd8554c20f10025052'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:12.643617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d19c0e43-79d8-41fb-9735-e3d5ba5204cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c171dee265246cd8554c20f10025052'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:12.656348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1292/2555] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:12 2022] DELETE /identity/v3/users/0c171dee265246cd8554c20f10025052 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:13.073499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:13.074946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:13.088103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b206f0-644f-4d97-9033-af4f80c8a558 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:13.144956 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:13.145584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:13.145857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:13.146167 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] PATH_INFO: `/v3/projects/713981f7dbdc4e5195cbf59e946e9d31` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:13.147280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=713981f7dbdc4e5195cbf59e946e9d31)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:13.149162 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:13.233502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:13.247798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '713981f7dbdc4e5195cbf59e946e9d31', 'request_id': 'req-cd40ea4f-44fe-4183-8959-601c1cd92f13'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:13.253906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '713981f7dbdc4e5195cbf59e946e9d31', 'request_id': 'req-cd40ea4f-44fe-4183-8959-601c1cd92f13'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:13.293735 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '713981f7dbdc4e5195cbf59e946e9d31', 'request_id': 'req-cd40ea4f-44fe-4183-8959-601c1cd92f13'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:13.304431 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] The token cache is being invalidate because project 713981f7dbdc4e5195cbf59e946e9d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:13.304722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:13.305666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-cd40ea4f-44fe-4183-8959-601c1cd92f13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:13.307635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1264/2556] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:13 2022] DELETE /identity/v3/projects/713981f7dbdc4e5195cbf59e946e9d31 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:13.313480 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52efa4-cd97-4ba3-8ddb-018c6dbd43b8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:13.313955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52efa4-cd97-4ba3-8ddb-018c6dbd43b8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:13.314088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52efa4-cd97-4ba3-8ddb-018c6dbd43b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:13.334268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6c52efa4-cd97-4ba3-8ddb-018c6dbd43b8 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:13.335470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6c52efa4-cd97-4ba3-8ddb-018c6dbd43b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:13.392913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1293/2557] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:13.400808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bb4a1-b086-43e3-b031-c73777bbe204 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:13.401093 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bb4a1-b086-43e3-b031-c73777bbe204 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:13.401368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bb4a1-b086-43e3-b031-c73777bbe204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:13.421023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-377bb4a1-b086-43e3-b031-c73777bbe204 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:13.422247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-377bb4a1-b086-43e3-b031-c73777bbe204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:13.430286 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1265/2558] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:13.434506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52efa4-cd97-4ba3-8ddb-018c6dbd43b8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:13.444361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52efa4-cd97-4ba3-8ddb-018c6dbd43b8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:13.449006 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253de1a0-fc47-4863-8e5f-cf5813a5f4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:13.449620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253de1a0-fc47-4863-8e5f-cf5813a5f4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:13.449905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253de1a0-fc47-4863-8e5f-cf5813a5f4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:13.450202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253de1a0-fc47-4863-8e5f-cf5813a5f4a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:13.450880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253de1a0-fc47-4863-8e5f-cf5813a5f4a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:50:13.451184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253de1a0-fc47-4863-8e5f-cf5813a5f4a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:13.453438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253de1a0-fc47-4863-8e5f-cf5813a5f4a6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:13.466698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1294/2559] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:50:13.471635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bb4a1-b086-43e3-b031-c73777bbe204 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:13.481561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bb4a1-b086-43e3-b031-c73777bbe204 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:13.486280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d308af6-dbb6-4efa-9663-3c35aec930cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:13.486793 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d308af6-dbb6-4efa-9663-3c35aec930cf admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:13.487075 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d308af6-dbb6-4efa-9663-3c35aec930cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:13.487446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d308af6-dbb6-4efa-9663-3c35aec930cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:13.488095 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d308af6-dbb6-4efa-9663-3c35aec930cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:13.490188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d308af6-dbb6-4efa-9663-3c35aec930cf admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:13.592032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1266/2560] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:13 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:13.596087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253de1a0-fc47-4863-8e5f-cf5813a5f4a6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:13.605964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253de1a0-fc47-4863-8e5f-cf5813a5f4a6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:13.631554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:13.661977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d56184-ec3d-46a3-8463-bfa6bb240ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:13.662615 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d56184-ec3d-46a3-8463-bfa6bb240ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:13.662894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d56184-ec3d-46a3-8463-bfa6bb240ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:13.663164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d56184-ec3d-46a3-8463-bfa6bb240ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:13.663910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d56184-ec3d-46a3-8463-bfa6bb240ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:13.665856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d56184-ec3d-46a3-8463-bfa6bb240ab6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:13.728311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1295/2561] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:13 2022] POST /identity/v3/users => generated 283 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:13.732353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d308af6-dbb6-4efa-9663-3c35aec930cf admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:13.742652 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d308af6-dbb6-4efa-9663-3c35aec930cf admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:13.747245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689e6d3-c51f-4a9a-954f-89e7418cb557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:13.747836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689e6d3-c51f-4a9a-954f-89e7418cb557 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:13.748115 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689e6d3-c51f-4a9a-954f-89e7418cb557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:13.748383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689e6d3-c51f-4a9a-954f-89e7418cb557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:13.749013 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689e6d3-c51f-4a9a-954f-89e7418cb557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:13.751109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689e6d3-c51f-4a9a-954f-89e7418cb557 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:13.760746 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1267/2562] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:13.764727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d56184-ec3d-46a3-8463-bfa6bb240ab6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:13.775493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d56184-ec3d-46a3-8463-bfa6bb240ab6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:13.780264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4397b-8285-40ad-bbdf-cddb1e2bcdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:13.781349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4397b-8285-40ad-bbdf-cddb1e2bcdd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:13.781349 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4397b-8285-40ad-bbdf-cddb1e2bcdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:13.781544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4397b-8285-40ad-bbdf-cddb1e2bcdd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:13.782038 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4397b-8285-40ad-bbdf-cddb1e2bcdd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:13.783659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4397b-8285-40ad-bbdf-cddb1e2bcdd1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:13.793417 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1296/2563] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:13.797680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689e6d3-c51f-4a9a-954f-89e7418cb557 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:13.807676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689e6d3-c51f-4a9a-954f-89e7418cb557 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:13.812334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309265e-8e7c-4a8e-880b-84d29b8a899d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:13.813469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309265e-8e7c-4a8e-880b-84d29b8a899d admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:13.813750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309265e-8e7c-4a8e-880b-84d29b8a899d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:13.814023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309265e-8e7c-4a8e-880b-84d29b8a899d admin admin] PATH_INFO: `/v3/projects/cf249d61ef5549fe8c9ead7763436614/users/b0b9f106591a417fbbfe98947dbeae4c/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:13.829062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309265e-8e7c-4a8e-880b-84d29b8a899d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf249d61ef5549fe8c9ead7763436614, user_id=b0b9f106591a417fbbfe98947dbeae4c, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:13.831732 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309265e-8e7c-4a8e-880b-84d29b8a899d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:13.877012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1268/2564] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:13 2022] PUT /identity/v3/projects/cf249d61ef5549fe8c9ead7763436614/users/b0b9f106591a417fbbfe98947dbeae4c/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:13.940990 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e82d9-cb2b-4bd5-82b9-d4267e613339 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:13.941545 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e82d9-cb2b-4bd5-82b9-d4267e613339 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:13.941825 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e82d9-cb2b-4bd5-82b9-d4267e613339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:13.969209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f23e82d9-cb2b-4bd5-82b9-d4267e613339 None None] MFA Rules not processed for user `b0b9f106591a417fbbfe98947dbeae4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:13.970461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f23e82d9-cb2b-4bd5-82b9-d4267e613339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:14.032853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1297/2565] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:14.044446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309265e-8e7c-4a8e-880b-84d29b8a899d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:14.046200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3309265e-8e7c-4a8e-880b-84d29b8a899d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:14.103047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309265e-8e7c-4a8e-880b-84d29b8a899d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:14.106914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77026081-bfec-4790-a0c0-005a90ad4e6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34eb8fcef0543c282b3887dfeb74c15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:14.107515 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77026081-bfec-4790-a0c0-005a90ad4e6b service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:14.107797 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77026081-bfec-4790-a0c0-005a90ad4e6b service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:14.108080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77026081-bfec-4790-a0c0-005a90ad4e6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:14.116901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77026081-bfec-4790-a0c0-005a90ad4e6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:14.118353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77026081-bfec-4790-a0c0-005a90ad4e6b service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:14.127636 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1269/2566] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:16.597746 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e82d9-cb2b-4bd5-82b9-d4267e613339 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:16.599156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f23e82d9-cb2b-4bd5-82b9-d4267e613339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:16.657726 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e82d9-cb2b-4bd5-82b9-d4267e613339 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:16.661314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca527cf-8633-40c1-98ed-f92a6c2bb413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:16.661910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca527cf-8633-40c1-98ed-f92a6c2bb413 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:16.662216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca527cf-8633-40c1-98ed-f92a6c2bb413 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:16.662493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca527cf-8633-40c1-98ed-f92a6c2bb413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:16.664262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fca527cf-8633-40c1-98ed-f92a6c2bb413 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:16.717880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:16.734976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77026081-bfec-4790-a0c0-005a90ad4e6b service glance] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:16.735521 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca527cf-8633-40c1-98ed-f92a6c2bb413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:16.736451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-77026081-bfec-4790-a0c0-005a90ad4e6b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:16.737018 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca527cf-8633-40c1-98ed-f92a6c2bb413 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:16.749964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1298/2567] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:16.754519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77026081-bfec-4790-a0c0-005a90ad4e6b service glance] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:16.758420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885de04-0094-48e7-8d75-3e07b6d8ee95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:16.759141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885de04-0094-48e7-8d75-3e07b6d8ee95 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:16.759495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885de04-0094-48e7-8d75-3e07b6d8ee95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:16.759776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885de04-0094-48e7-8d75-3e07b6d8ee95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:16.761733 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8885de04-0094-48e7-8d75-3e07b6d8ee95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:16.828847 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:16.843587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885de04-0094-48e7-8d75-3e07b6d8ee95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:16.845101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885de04-0094-48e7-8d75-3e07b6d8ee95 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:16.855304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1270/2568] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:27.066602 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:27.180139 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:43.929901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca527cf-8633-40c1-98ed-f92a6c2bb413 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:43.931430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fca527cf-8633-40c1-98ed-f92a6c2bb413 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:44.081048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca527cf-8633-40c1-98ed-f92a6c2bb413 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:44.086397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:44.087117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:44.087468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:44.087751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] PATH_INFO: `/v3/users/5e4a79b8857f40b8bd9e28e390148066` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:44.088907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4a79b8857f40b8bd9e28e390148066)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:44.090870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:44.231715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885de04-0094-48e7-8d75-3e07b6d8ee95 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:44.231715 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8885de04-0094-48e7-8d75-3e07b6d8ee95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:44.280371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885de04-0094-48e7-8d75-3e07b6d8ee95 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:44.285631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:44.286307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:44.286583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:44.286848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] PATH_INFO: `/v3/users/e1ad71c0bab546788d2d2468165d1b04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:44.287961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ad71c0bab546788d2d2468165d1b04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:44.289830 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:44.291645 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4a79b8857f40b8bd9e28e390148066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.313650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4a79b8857f40b8bd9e28e390148066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.314571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4a79b8857f40b8bd9e28e390148066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.408228 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4a79b8857f40b8bd9e28e390148066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.409745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-dbfe4b7c-0d94-44bf-a89b-2c4d6d6f9b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4a79b8857f40b8bd9e28e390148066'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.423562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1299/2569] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:43 2022] DELETE /identity/v3/users/5e4a79b8857f40b8bd9e28e390148066 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:44.499717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ad71c0bab546788d2d2468165d1b04'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.500510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ad71c0bab546788d2d2468165d1b04'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.514114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ad71c0bab546788d2d2468165d1b04'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.631269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ad71c0bab546788d2d2468165d1b04'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.631914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ad71c0bab546788d2d2468165d1b04'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.646301 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1271/2570] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:44 2022] DELETE /identity/v3/users/e1ad71c0bab546788d2d2468165d1b04 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:44.650827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455047-8c9e-4b3c-be94-3d25b3848c24 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:44.651149 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455047-8c9e-4b3c-be94-3d25b3848c24 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:44.651428 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca455047-8c9e-4b3c-be94-3d25b3848c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:44.679659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ca455047-8c9e-4b3c-be94-3d25b3848c24 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:44.680891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ca455047-8c9e-4b3c-be94-3d25b3848c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:44.686021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:44.690904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:44.692800 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:44.750762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1300/2571] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:44.761732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca455047-8c9e-4b3c-be94-3d25b3848c24 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:44.763649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ca455047-8c9e-4b3c-be94-3d25b3848c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:44.775479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a874-a820-49a8-a4da-99b31a3959cc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:44.780050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:44.780678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:44.780951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:44.781224 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] PATH_INFO: `/v3/projects/8207d8781edd40f3b4087a601696d5f3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:44.782365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8207d8781edd40f3b4087a601696d5f3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:44.784984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:44.837885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca455047-8c9e-4b3c-be94-3d25b3848c24 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:44.841616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074c447-cda9-4a80-888b-2d9b9a85cd87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:44.842249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074c447-cda9-4a80-888b-2d9b9a85cd87 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:44.842532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074c447-cda9-4a80-888b-2d9b9a85cd87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:44.842817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074c447-cda9-4a80-888b-2d9b9a85cd87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:44.852256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074c447-cda9-4a80-888b-2d9b9a85cd87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:44.853683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074c447-cda9-4a80-888b-2d9b9a85cd87 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:44.863074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1301/2572] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:44.915215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:44.922449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8207d8781edd40f3b4087a601696d5f3', 'request_id': 'req-acdb44eb-817c-4e74-b96f-2925c3f54d34'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:44.928865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8207d8781edd40f3b4087a601696d5f3', 'request_id': 'req-acdb44eb-817c-4e74-b96f-2925c3f54d34'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.032264 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8207d8781edd40f3b4087a601696d5f3', 'request_id': 'req-acdb44eb-817c-4e74-b96f-2925c3f54d34'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.043791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] The token cache is being invalidate because project 8207d8781edd40f3b4087a601696d5f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:45.044079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.044733 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.046413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1272/2573] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:44 2022] DELETE /identity/v3/projects/8207d8781edd40f3b4087a601696d5f3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:45.050839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074c447-cda9-4a80-888b-2d9b9a85cd87 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:45.060141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074c447-cda9-4a80-888b-2d9b9a85cd87 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:45.153737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:45.153737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:45.153737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:45.153737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] PATH_INFO: `/v3/users/f4480b1b068c48cb93f789dd22a6766b` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:45.154091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4480b1b068c48cb93f789dd22a6766b)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:45.154091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:45.268813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:45.270495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:45.289048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb44eb-817c-4e74-b96f-2925c3f54d34 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:45.301166 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:45.301852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:45.302332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:45.302458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] PATH_INFO: `/v3/projects/1d054fe5fc3648de8cb9be51aaec187c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:45.303575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d054fe5fc3648de8cb9be51aaec187c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:45.305462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:45.339487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:45.344648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4480b1b068c48cb93f789dd22a6766b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.356808 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4480b1b068c48cb93f789dd22a6766b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.357487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4480b1b068c48cb93f789dd22a6766b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.402439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4480b1b068c48cb93f789dd22a6766b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.403296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4480b1b068c48cb93f789dd22a6766b'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.404979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d054fe5fc3648de8cb9be51aaec187c', 'request_id': 'req-28ca2260-1481-4b0b-9145-4aa2b305f9f3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.411786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d054fe5fc3648de8cb9be51aaec187c', 'request_id': 'req-28ca2260-1481-4b0b-9145-4aa2b305f9f3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.419274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1302/2574] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:45 2022] DELETE /identity/v3/users/f4480b1b068c48cb93f789dd22a6766b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:45.466219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d054fe5fc3648de8cb9be51aaec187c', 'request_id': 'req-28ca2260-1481-4b0b-9145-4aa2b305f9f3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.477617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] The token cache is being invalidate because project 1d054fe5fc3648de8cb9be51aaec187c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:45.478014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.478709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-28ca2260-1481-4b0b-9145-4aa2b305f9f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.480215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1273/2575] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:45 2022] DELETE /identity/v3/projects/1d054fe5fc3648de8cb9be51aaec187c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:45.484255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:45.485689 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:45.570999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526555c9-8cab-4856-a858-ee067a211dfc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:45.570999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:45.570999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:45.570999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:45.571326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] PATH_INFO: `/v3/users/c8d23412b87f4f4ca037b3866be14653` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:45.571326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d23412b87f4f4ca037b3866be14653)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:45.572932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:45.574927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ce30c-d4e6-4814-9f9b-177c5d9c5139 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:45.575206 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ce30c-d4e6-4814-9f9b-177c5d9c5139 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:45.575757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ce30c-d4e6-4814-9f9b-177c5d9c5139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:45.597662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d04ce30c-d4e6-4814-9f9b-177c5d9c5139 None None] MFA Rules not processed for user `0c0c0a25340a451b970cc3de8bc05806`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:45.598922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d04ce30c-d4e6-4814-9f9b-177c5d9c5139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:45.661261 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1274/2576] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:45.666517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ce30c-d4e6-4814-9f9b-177c5d9c5139 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:45.668210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d04ce30c-d4e6-4814-9f9b-177c5d9c5139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:45.692999 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ce30c-d4e6-4814-9f9b-177c5d9c5139 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:45.698210 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:45.698879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:45.699169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:45.699451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] PATH_INFO: `/v3/projects/416c5642950a48d9a2faeadc45f6e8a4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:45.700641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=416c5642950a48d9a2faeadc45f6e8a4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:45.703202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:45.759904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d23412b87f4f4ca037b3866be14653'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.773323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d23412b87f4f4ca037b3866be14653'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.774285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d23412b87f4f4ca037b3866be14653'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.818240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '416c5642950a48d9a2faeadc45f6e8a4', 'request_id': 'req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.821109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:45.821972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d23412b87f4f4ca037b3866be14653'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.822633 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f5f4de3d-0fb1-439c-a882-4efd0b859df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d23412b87f4f4ca037b3866be14653'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.827722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '416c5642950a48d9a2faeadc45f6e8a4', 'request_id': 'req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.837396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1303/2577] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:45 2022] DELETE /identity/v3/users/c8d23412b87f4f4ca037b3866be14653 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:45.911414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '416c5642950a48d9a2faeadc45f6e8a4', 'request_id': 'req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.923668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] The token cache is being invalidate because project 416c5642950a48d9a2faeadc45f6e8a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:45.924669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.925447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-997bfa9b-2c3a-4c54-8527-d4b0f4bae562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:45.931419 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1275/2578] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:45 2022] DELETE /identity/v3/projects/416c5642950a48d9a2faeadc45f6e8a4 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:45.933084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4af02-9080-42f0-8aca-56c9f4a613b1 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:45.933404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4af02-9080-42f0-8aca-56c9f4a613b1 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:45.933650 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4af02-9080-42f0-8aca-56c9f4a613b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:45.984467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2cf4af02-9080-42f0-8aca-56c9f4a613b1 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:45.985717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2cf4af02-9080-42f0-8aca-56c9f4a613b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:45.994977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:46.051658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1304/2579] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:46.058805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da775b-5346-461e-9ca1-4b4ad227fab2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.059097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da775b-5346-461e-9ca1-4b4ad227fab2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.059376 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da775b-5346-461e-9ca1-4b4ad227fab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.082823 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d1da775b-5346-461e-9ca1-4b4ad227fab2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:46.084149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d1da775b-5346-461e-9ca1-4b4ad227fab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:46.092546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1276/2580] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:46.097626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4af02-9080-42f0-8aca-56c9f4a613b1 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.107307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4af02-9080-42f0-8aca-56c9f4a613b1 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.112101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb23f4-70cc-4851-9bef-3d10f42828fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:46.112703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb23f4-70cc-4851-9bef-3d10f42828fe admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.112979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb23f4-70cc-4851-9bef-3d10f42828fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.113250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb23f4-70cc-4851-9bef-3d10f42828fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.113916 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb23f4-70cc-4851-9bef-3d10f42828fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:50:46.114263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb23f4-70cc-4851-9bef-3d10f42828fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:46.115932 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb23f4-70cc-4851-9bef-3d10f42828fe admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:46.132865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1305/2581] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:50:46.133015 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da775b-5346-461e-9ca1-4b4ad227fab2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.142241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da775b-5346-461e-9ca1-4b4ad227fab2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.147906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026c0f6-b991-4cf9-8643-7ad44b11aa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:46.148641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026c0f6-b991-4cf9-8643-7ad44b11aa72 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.148986 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026c0f6-b991-4cf9-8643-7ad44b11aa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.149327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026c0f6-b991-4cf9-8643-7ad44b11aa72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.150127 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026c0f6-b991-4cf9-8643-7ad44b11aa72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:46.152152 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026c0f6-b991-4cf9-8643-7ad44b11aa72 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:46.239104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1277/2582] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:46 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:46.243156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb23f4-70cc-4851-9bef-3d10f42828fe admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.253340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb23f4-70cc-4851-9bef-3d10f42828fe admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.311759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e91747-ef5f-451e-8996-07a854c874a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:46.312368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e91747-ef5f-451e-8996-07a854c874a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.312649 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e91747-ef5f-451e-8996-07a854c874a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.312918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e91747-ef5f-451e-8996-07a854c874a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.313552 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e91747-ef5f-451e-8996-07a854c874a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:46.315515 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e91747-ef5f-451e-8996-07a854c874a4 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:46.380775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026c0f6-b991-4cf9-8643-7ad44b11aa72 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.380775 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d026c0f6-b991-4cf9-8643-7ad44b11aa72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:46.395003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026c0f6-b991-4cf9-8643-7ad44b11aa72 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.401688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:46.402484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.403036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.403551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] PATH_INFO: `/v3/projects/636860aaeaa14048883741791d4c92e1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.404933 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=636860aaeaa14048883741791d4c92e1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:46.407178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:46.408017 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1306/2583] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:46 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:46.414698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e91747-ef5f-451e-8996-07a854c874a4 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.426771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e91747-ef5f-451e-8996-07a854c874a4 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.431585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2ce18-97e2-49f0-b7d2-dd6e9728f6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:46.432184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2ce18-97e2-49f0-b7d2-dd6e9728f6aa admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.432461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2ce18-97e2-49f0-b7d2-dd6e9728f6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.432741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2ce18-97e2-49f0-b7d2-dd6e9728f6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.433370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2ce18-97e2-49f0-b7d2-dd6e9728f6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:46.434987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2ce18-97e2-49f0-b7d2-dd6e9728f6aa admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:46.448909 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1307/2584] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:46.453007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2ce18-97e2-49f0-b7d2-dd6e9728f6aa admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.463774 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2ce18-97e2-49f0-b7d2-dd6e9728f6aa admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.469595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c68be-440a-40e6-9c2f-006c5407fb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:46.470236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c68be-440a-40e6-9c2f-006c5407fb6a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.470518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c68be-440a-40e6-9c2f-006c5407fb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.470811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c68be-440a-40e6-9c2f-006c5407fb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.471458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c68be-440a-40e6-9c2f-006c5407fb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:46.473056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c68be-440a-40e6-9c2f-006c5407fb6a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:46.484070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1308/2585] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:46.496190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c68be-440a-40e6-9c2f-006c5407fb6a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.507061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c68be-440a-40e6-9c2f-006c5407fb6a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.511752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c218e99-bb00-4148-8c5e-b096c2fa68fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:46.512385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c218e99-bb00-4148-8c5e-b096c2fa68fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.512664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c218e99-bb00-4148-8c5e-b096c2fa68fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.512938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c218e99-bb00-4148-8c5e-b096c2fa68fc admin admin] PATH_INFO: `/v3/projects/c286b5514e7f4da4a6a9377b7cc257c4/users/132dd0a1b18d4083a5aa0bb174d383f7/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.528069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c218e99-bb00-4148-8c5e-b096c2fa68fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c286b5514e7f4da4a6a9377b7cc257c4, user_id=132dd0a1b18d4083a5aa0bb174d383f7, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:46.530818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c218e99-bb00-4148-8c5e-b096c2fa68fc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:46.614352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1309/2586] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:46 2022] PUT /identity/v3/projects/c286b5514e7f4da4a6a9377b7cc257c4/users/132dd0a1b18d4083a5aa0bb174d383f7/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:46.623027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636860aaeaa14048883741791d4c92e1', 'request_id': 'req-f9fce55e-71a2-48ae-bbab-b802185b3457'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:46.624692 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecade214-ecab-4fa7-bd0e-50ec0b4691c2 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.624978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecade214-ecab-4fa7-bd0e-50ec0b4691c2 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.625257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecade214-ecab-4fa7-bd0e-50ec0b4691c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.630726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636860aaeaa14048883741791d4c92e1', 'request_id': 'req-f9fce55e-71a2-48ae-bbab-b802185b3457'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:46.653299 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ecade214-ecab-4fa7-bd0e-50ec0b4691c2 None None] MFA Rules not processed for user `132dd0a1b18d4083a5aa0bb174d383f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:46.654551 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ecade214-ecab-4fa7-bd0e-50ec0b4691c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:46.688360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636860aaeaa14048883741791d4c92e1', 'request_id': 'req-f9fce55e-71a2-48ae-bbab-b802185b3457'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:46.690065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:46.706581 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] The token cache is being invalidate because project 636860aaeaa14048883741791d4c92e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:46.706898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:46.707585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-f9fce55e-71a2-48ae-bbab-b802185b3457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:46.709131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1278/2587] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:46 2022] DELETE /identity/v3/projects/636860aaeaa14048883741791d4c92e1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:46.714065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3dabd-fd14-4b6d-9505-01bff79cd000 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.714375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3dabd-fd14-4b6d-9505-01bff79cd000 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.714651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3dabd-fd14-4b6d-9505-01bff79cd000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.725204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1310/2588] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:46.729751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecade214-ecab-4fa7-bd0e-50ec0b4691c2 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.731104 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ecade214-ecab-4fa7-bd0e-50ec0b4691c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:46.736231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-98c3dabd-fd14-4b6d-9505-01bff79cd000 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:46.737437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-98c3dabd-fd14-4b6d-9505-01bff79cd000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:46.805107 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecade214-ecab-4fa7-bd0e-50ec0b4691c2 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.810780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326608f-3cc1-49f0-a4ec-34135fe0893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:46.811468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326608f-3cc1-49f0-a4ec-34135fe0893e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.812082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326608f-3cc1-49f0-a4ec-34135fe0893e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.812377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326608f-3cc1-49f0-a4ec-34135fe0893e admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.813353 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326608f-3cc1-49f0-a4ec-34135fe0893e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:46.816038 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326608f-3cc1-49f0-a4ec-34135fe0893e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:46.832827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1279/2589] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:46.834785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29a142-f323-43c0-88d8-ff17a34cfb81 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.835183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29a142-f323-43c0-88d8-ff17a34cfb81 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.835462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29a142-f323-43c0-88d8-ff17a34cfb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.857125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-eb29a142-f323-43c0-88d8-ff17a34cfb81 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:46.859509 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-eb29a142-f323-43c0-88d8-ff17a34cfb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:46.869051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1280/2590] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:46.873566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29a142-f323-43c0-88d8-ff17a34cfb81 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.884323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29a142-f323-43c0-88d8-ff17a34cfb81 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.895333 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaccad-c65a-4781-ba39-54c628aa730f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:46.898248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaccad-c65a-4781-ba39-54c628aa730f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.898973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaccad-c65a-4781-ba39-54c628aa730f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.899255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaccad-c65a-4781-ba39-54c628aa730f admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.899959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaaccad-c65a-4781-ba39-54c628aa730f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:50:46.900324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaaccad-c65a-4781-ba39-54c628aa730f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:46.901925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaaccad-c65a-4781-ba39-54c628aa730f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:46.910188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1311/2591] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:46 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:46.913532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326608f-3cc1-49f0-a4ec-34135fe0893e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.918388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1281/2592] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:50:46.919600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaccad-c65a-4781-ba39-54c628aa730f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:46.936951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaccad-c65a-4781-ba39-54c628aa730f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.952742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:46.987593 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326608f-3cc1-49f0-a4ec-34135fe0893e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:46.993906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc359e-f391-4430-bd76-5dee59b161b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:46.994559 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc359e-f391-4430-bd76-5dee59b161b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:46.995010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc359e-f391-4430-bd76-5dee59b161b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:46.995554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc359e-f391-4430-bd76-5dee59b161b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:46.996370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc359e-f391-4430-bd76-5dee59b161b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:46.998880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc359e-f391-4430-bd76-5dee59b161b6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.005228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615fd51-7bab-4812-8490-317a5bd1e1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:47.005866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615fd51-7bab-4812-8490-317a5bd1e1ff admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.006156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615fd51-7bab-4812-8490-317a5bd1e1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.006440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615fd51-7bab-4812-8490-317a5bd1e1ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.007066 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615fd51-7bab-4812-8490-317a5bd1e1ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:47.008953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615fd51-7bab-4812-8490-317a5bd1e1ff admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.137254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1312/2593] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:46 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:47.137786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1282/2594] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:46 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:47.138426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc359e-f391-4430-bd76-5dee59b161b6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:47.140906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615fd51-7bab-4812-8490-317a5bd1e1ff admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:47.149897 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc359e-f391-4430-bd76-5dee59b161b6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:47.199991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:47.209781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615fd51-7bab-4812-8490-317a5bd1e1ff admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:47.212753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562221a3-cdbd-4e2f-9ae9-479c46583d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:47.213740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562221a3-cdbd-4e2f-9ae9-479c46583d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.214318 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562221a3-cdbd-4e2f-9ae9-479c46583d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.214629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66982553-6e41-4aa9-a4ee-9c2b36279e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:47.214988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562221a3-cdbd-4e2f-9ae9-479c46583d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.215277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66982553-6e41-4aa9-a4ee-9c2b36279e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.215560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66982553-6e41-4aa9-a4ee-9c2b36279e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.215836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66982553-6e41-4aa9-a4ee-9c2b36279e70 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.215951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562221a3-cdbd-4e2f-9ae9-479c46583d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:47.216465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66982553-6e41-4aa9-a4ee-9c2b36279e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:47.217579 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562221a3-cdbd-4e2f-9ae9-479c46583d5c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.218393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66982553-6e41-4aa9-a4ee-9c2b36279e70 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.228789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1313/2595] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:47.232659 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562221a3-cdbd-4e2f-9ae9-479c46583d5c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:47.243876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562221a3-cdbd-4e2f-9ae9-479c46583d5c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:47.248569 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465a245-a6bd-47c2-a5e2-2bcae072d246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:47.249163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465a245-a6bd-47c2-a5e2-2bcae072d246 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.249437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465a245-a6bd-47c2-a5e2-2bcae072d246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.249706 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465a245-a6bd-47c2-a5e2-2bcae072d246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.250373 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465a245-a6bd-47c2-a5e2-2bcae072d246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:47.251947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465a245-a6bd-47c2-a5e2-2bcae072d246 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.262565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1314/2596] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:47.266747 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465a245-a6bd-47c2-a5e2-2bcae072d246 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:47.275901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465a245-a6bd-47c2-a5e2-2bcae072d246 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:47.280358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb52748-6cdb-4b8a-b20e-aca159dc58e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:47.280960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb52748-6cdb-4b8a-b20e-aca159dc58e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.281233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb52748-6cdb-4b8a-b20e-aca159dc58e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.281504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb52748-6cdb-4b8a-b20e-aca159dc58e3 admin admin] PATH_INFO: `/v3/projects/7ea33ee304ac4545898f04bc988da18c/users/54f5ddeac6d347259b4ecc5efb2d730f/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.295971 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb52748-6cdb-4b8a-b20e-aca159dc58e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea33ee304ac4545898f04bc988da18c, user_id=54f5ddeac6d347259b4ecc5efb2d730f, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:47.298699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb52748-6cdb-4b8a-b20e-aca159dc58e3 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.328849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1283/2597] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:47 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:47.332656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66982553-6e41-4aa9-a4ee-9c2b36279e70 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:47.342076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66982553-6e41-4aa9-a4ee-9c2b36279e70 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:47.347346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f280d-6049-4a57-901c-b320e8794270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:47.347930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f280d-6049-4a57-901c-b320e8794270 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.348208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f280d-6049-4a57-901c-b320e8794270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.348505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f280d-6049-4a57-901c-b320e8794270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.349136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f280d-6049-4a57-901c-b320e8794270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:47.350751 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f280d-6049-4a57-901c-b320e8794270 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.362761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1284/2598] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:47.366421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f280d-6049-4a57-901c-b320e8794270 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:47.376088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f280d-6049-4a57-901c-b320e8794270 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:47.380861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723dd00-7887-4951-8bd9-11063178e72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:47.381459 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723dd00-7887-4951-8bd9-11063178e72e admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.381656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723dd00-7887-4951-8bd9-11063178e72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.381926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723dd00-7887-4951-8bd9-11063178e72e admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.382632 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723dd00-7887-4951-8bd9-11063178e72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:47.384222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723dd00-7887-4951-8bd9-11063178e72e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.398782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1285/2599] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:47.400391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723dd00-7887-4951-8bd9-11063178e72e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:47.411008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723dd00-7887-4951-8bd9-11063178e72e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:47.415665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046388d4-58e8-4e39-84e2-334e293e5bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:47.416315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046388d4-58e8-4e39-84e2-334e293e5bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.416590 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046388d4-58e8-4e39-84e2-334e293e5bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.416860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046388d4-58e8-4e39-84e2-334e293e5bc4 admin admin] PATH_INFO: `/v3/projects/e59f33ee9d9a442688de65d7ee796701/users/42fce22f3a464a69a4c385540ad9e9d2/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.427136 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1315/2600] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:47 2022] PUT /identity/v3/projects/7ea33ee304ac4545898f04bc988da18c/users/54f5ddeac6d347259b4ecc5efb2d730f/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:47.433543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046388d4-58e8-4e39-84e2-334e293e5bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e59f33ee9d9a442688de65d7ee796701, user_id=42fce22f3a464a69a4c385540ad9e9d2, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:47.436228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046388d4-58e8-4e39-84e2-334e293e5bc4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.439724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648913ed-5133-4fb0-b3f4-449d37b5201f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.440310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648913ed-5133-4fb0-b3f4-449d37b5201f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.440619 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648913ed-5133-4fb0-b3f4-449d37b5201f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.468347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-648913ed-5133-4fb0-b3f4-449d37b5201f None None] MFA Rules not processed for user `54f5ddeac6d347259b4ecc5efb2d730f`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:47.469571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-648913ed-5133-4fb0-b3f4-449d37b5201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:47.507049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1286/2601] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:47 2022] PUT /identity/v3/projects/e59f33ee9d9a442688de65d7ee796701/users/42fce22f3a464a69a4c385540ad9e9d2/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:47.521094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b80db-948c-4535-9343-21b2249cb98d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.521506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b80db-948c-4535-9343-21b2249cb98d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.521871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b80db-948c-4535-9343-21b2249cb98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.540680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1316/2602] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:47.551244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648913ed-5133-4fb0-b3f4-449d37b5201f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:47.557521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-dd8b80db-948c-4535-9343-21b2249cb98d None None] MFA Rules not processed for user `42fce22f3a464a69a4c385540ad9e9d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:47.558765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-dd8b80db-948c-4535-9343-21b2249cb98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:47.594795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:47.619332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648913ed-5133-4fb0-b3f4-449d37b5201f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:47.627839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1287/2603] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:47.632469 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b80db-948c-4535-9343-21b2249cb98d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:47.666061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:47.678702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed69b6-2d4a-464d-830c-a759dc74f18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:47.679409 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed69b6-2d4a-464d-830c-a759dc74f18c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.679587 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed69b6-2d4a-464d-830c-a759dc74f18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.679861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed69b6-2d4a-464d-830c-a759dc74f18c admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.680489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed69b6-2d4a-464d-830c-a759dc74f18c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:47.682431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed69b6-2d4a-464d-830c-a759dc74f18c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.694763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b80db-948c-4535-9343-21b2249cb98d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:47.700402 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7838e-8821-404d-8abd-dc817aa30320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:47.701030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7838e-8821-404d-8abd-dc817aa30320 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:47.701312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7838e-8821-404d-8abd-dc817aa30320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:47.701639 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7838e-8821-404d-8abd-dc817aa30320 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:47.702324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7838e-8821-404d-8abd-dc817aa30320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:47.704375 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7838e-8821-404d-8abd-dc817aa30320 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:47.931003 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1288/2604] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:47 2022] POST /identity/v3/projects => generated 379 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:47.935082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7838e-8821-404d-8abd-dc817aa30320 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:47.935522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1317/2605] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:47 2022] POST /identity/v3/projects => generated 375 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:47.939090 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed69b6-2d4a-464d-830c-a759dc74f18c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.013867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7838e-8821-404d-8abd-dc817aa30320 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.018979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33efa-bdbf-4c51-baed-c0bda475bc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.019734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33efa-bdbf-4c51-baed-c0bda475bc96 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.020008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33efa-bdbf-4c51-baed-c0bda475bc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.020278 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33efa-bdbf-4c51-baed-c0bda475bc96 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.020899 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33efa-bdbf-4c51-baed-c0bda475bc96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.022864 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33efa-bdbf-4c51-baed-c0bda475bc96 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.025067 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed69b6-2d4a-464d-830c-a759dc74f18c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.030665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff62a35-1097-459a-9d89-9ae4c691c0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.031462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff62a35-1097-459a-9d89-9ae4c691c0ac admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.031763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff62a35-1097-459a-9d89-9ae4c691c0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.032416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff62a35-1097-459a-9d89-9ae4c691c0ac admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.033057 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff62a35-1097-459a-9d89-9ae4c691c0ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.035801 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff62a35-1097-459a-9d89-9ae4c691c0ac admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.114780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1289/2606] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:47 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:48.117041 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1318/2607] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:47 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:48.120618 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33efa-bdbf-4c51-baed-c0bda475bc96 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.120884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff62a35-1097-459a-9d89-9ae4c691c0ac admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.131432 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff62a35-1097-459a-9d89-9ae4c691c0ac admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.134114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33efa-bdbf-4c51-baed-c0bda475bc96 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.136167 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6ac5d-1c07-4d90-848d-4ca8a7caaf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.136323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fb940-40c5-4cb2-bcb2-3d38aa5bf6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.136762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6ac5d-1c07-4d90-848d-4ca8a7caaf12 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.136892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb940-40c5-4cb2-bcb2-3d38aa5bf6ea admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.137040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6ac5d-1c07-4d90-848d-4ca8a7caaf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.137168 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb940-40c5-4cb2-bcb2-3d38aa5bf6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.137366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6ac5d-1c07-4d90-848d-4ca8a7caaf12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.137475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb940-40c5-4cb2-bcb2-3d38aa5bf6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.137938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6ac5d-1c07-4d90-848d-4ca8a7caaf12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.138084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fb940-40c5-4cb2-bcb2-3d38aa5bf6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.139546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb6ac5d-1c07-4d90-848d-4ca8a7caaf12 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.139732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fb940-40c5-4cb2-bcb2-3d38aa5bf6ea admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.150190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1290/2608] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:48.150556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1319/2609] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:48.154598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6ac5d-1c07-4d90-848d-4ca8a7caaf12 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.154864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fb940-40c5-4cb2-bcb2-3d38aa5bf6ea admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.165503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fb940-40c5-4cb2-bcb2-3d38aa5bf6ea admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.165683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6ac5d-1c07-4d90-848d-4ca8a7caaf12 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.170015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dc9de-b999-4d17-b29f-2d0d1e141ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.170584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd792a-6a1b-4031-97c3-1f0a33c12305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.171263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd792a-6a1b-4031-97c3-1f0a33c12305 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.171396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dc9de-b999-4d17-b29f-2d0d1e141ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.171548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dc9de-b999-4d17-b29f-2d0d1e141ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.171671 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd792a-6a1b-4031-97c3-1f0a33c12305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.171825 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dc9de-b999-4d17-b29f-2d0d1e141ce7 admin admin] PATH_INFO: `/v3/projects/b0741ca2f5b14118b25959b859ee54c3/users/3d6ae59e363c43e3ab77c24f51f48b03/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.171943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd792a-6a1b-4031-97c3-1f0a33c12305 admin admin] PATH_INFO: `/v3/projects/0e97f38fb983416181d5267b7d92acb1/users/da7068d3ec9a4287b2dffbe2e8d40d47/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.187877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dc9de-b999-4d17-b29f-2d0d1e141ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0741ca2f5b14118b25959b859ee54c3, user_id=3d6ae59e363c43e3ab77c24f51f48b03, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.190652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dc9de-b999-4d17-b29f-2d0d1e141ce7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.191248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd792a-6a1b-4031-97c3-1f0a33c12305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e97f38fb983416181d5267b7d92acb1, user_id=da7068d3ec9a4287b2dffbe2e8d40d47, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.195425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd792a-6a1b-4031-97c3-1f0a33c12305 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.300468 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1291/2610] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:48 2022] PUT /identity/v3/projects/0e97f38fb983416181d5267b7d92acb1/users/da7068d3ec9a4287b2dffbe2e8d40d47/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:48.301413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1320/2611] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:48 2022] PUT /identity/v3/projects/b0741ca2f5b14118b25959b859ee54c3/users/3d6ae59e363c43e3ab77c24f51f48b03/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:48.304602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd792a-6a1b-4031-97c3-1f0a33c12305 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.305248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dc9de-b999-4d17-b29f-2d0d1e141ce7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.321754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:48.369195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dc9de-b999-4d17-b29f-2d0d1e141ce7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.372341 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd792a-6a1b-4031-97c3-1f0a33c12305 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.376303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17507a9-65f0-4f25-b6c2-5af9ec0827f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.376303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17507a9-65f0-4f25-b6c2-5af9ec0827f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.376303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17507a9-65f0-4f25-b6c2-5af9ec0827f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.376303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17507a9-65f0-4f25-b6c2-5af9ec0827f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.376915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17507a9-65f0-4f25-b6c2-5af9ec0827f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.377658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f387e17-aa70-4940-8a1a-959fbc1c3223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.378021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17507a9-65f0-4f25-b6c2-5af9ec0827f6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.378411 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387e17-aa70-4940-8a1a-959fbc1c3223 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.379037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387e17-aa70-4940-8a1a-959fbc1c3223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.379484 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387e17-aa70-4940-8a1a-959fbc1c3223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.380313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f387e17-aa70-4940-8a1a-959fbc1c3223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.381939 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f387e17-aa70-4940-8a1a-959fbc1c3223 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.392465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1321/2612] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:48.396716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1292/2613] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:48.396866 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17507a9-65f0-4f25-b6c2-5af9ec0827f6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.397670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f387e17-aa70-4940-8a1a-959fbc1c3223 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.407362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17507a9-65f0-4f25-b6c2-5af9ec0827f6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.408296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f387e17-aa70-4940-8a1a-959fbc1c3223 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.413662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e9ba9-4daf-4e4c-b767-6b7a4a0d97f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.413851 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855b915-0a9d-4d41-b796-11d367cf439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.414547 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855b915-0a9d-4d41-b796-11d367cf439f admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.414673 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e9ba9-4daf-4e4c-b767-6b7a4a0d97f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.415123 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855b915-0a9d-4d41-b796-11d367cf439f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.415252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e9ba9-4daf-4e4c-b767-6b7a4a0d97f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.415354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855b915-0a9d-4d41-b796-11d367cf439f admin admin] PATH_INFO: `/v3/projects/b0741ca2f5b14118b25959b859ee54c3/users/3d6ae59e363c43e3ab77c24f51f48b03/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.415463 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e9ba9-4daf-4e4c-b767-6b7a4a0d97f3 admin admin] PATH_INFO: `/v3/projects/0e97f38fb983416181d5267b7d92acb1/users/da7068d3ec9a4287b2dffbe2e8d40d47/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.417056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855b915-0a9d-4d41-b796-11d367cf439f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0741ca2f5b14118b25959b859ee54c3, user_id=3d6ae59e363c43e3ab77c24f51f48b03, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.417507 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e9ba9-4daf-4e4c-b767-6b7a4a0d97f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e97f38fb983416181d5267b7d92acb1, user_id=da7068d3ec9a4287b2dffbe2e8d40d47, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.419756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855b915-0a9d-4d41-b796-11d367cf439f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.420221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e9ba9-4daf-4e4c-b767-6b7a4a0d97f3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.430678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1322/2615] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:48 2022] PUT /identity/v3/projects/b0741ca2f5b14118b25959b859ee54c3/users/3d6ae59e363c43e3ab77c24f51f48b03/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:48.439622 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1293/2615] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:48 2022] PUT /identity/v3/projects/0e97f38fb983416181d5267b7d92acb1/users/da7068d3ec9a4287b2dffbe2e8d40d47/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:48.440815 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b66d9d-2383-463a-aa16-5e3d5cc3545c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.441109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b66d9d-2383-463a-aa16-5e3d5cc3545c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.441388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b66d9d-2383-463a-aa16-5e3d5cc3545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.442173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b6483-37f1-4c0d-ac51-f9b5cd23203d None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.442477 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b6483-37f1-4c0d-ac51-f9b5cd23203d None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.442810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b6483-37f1-4c0d-ac51-f9b5cd23203d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.464321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:48.474181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f2b66d9d-2383-463a-aa16-5e3d5cc3545c None None] MFA Rules not processed for user `3d6ae59e363c43e3ab77c24f51f48b03`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:48.475986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f2b66d9d-2383-463a-aa16-5e3d5cc3545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:48.480798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a14b6483-37f1-4c0d-ac51-f9b5cd23203d None None] MFA Rules not processed for user `da7068d3ec9a4287b2dffbe2e8d40d47`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:48.484449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a14b6483-37f1-4c0d-ac51-f9b5cd23203d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:48.561657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1323/2616] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:48.566607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1294/2617] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:48.578064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b66d9d-2383-463a-aa16-5e3d5cc3545c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.579596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f2b66d9d-2383-463a-aa16-5e3d5cc3545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:48.617826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b6483-37f1-4c0d-ac51-f9b5cd23203d None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:48.619524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a14b6483-37f1-4c0d-ac51-f9b5cd23203d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:48.645436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b66d9d-2383-463a-aa16-5e3d5cc3545c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.649250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251137f-ee3e-46de-b211-1a03ef706f61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.650235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251137f-ee3e-46de-b211-1a03ef706f61 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.650235 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251137f-ee3e-46de-b211-1a03ef706f61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.650478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251137f-ee3e-46de-b211-1a03ef706f61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.684516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b6483-37f1-4c0d-ac51-f9b5cd23203d None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:48.688779 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4ca0a-86dd-4335-bd2d-181b801e41b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:48.689421 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4ca0a-86dd-4335-bd2d-181b801e41b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:48.690247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4ca0a-86dd-4335-bd2d-181b801e41b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:48.690538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4ca0a-86dd-4335-bd2d-181b801e41b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:48.699734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4ca0a-86dd-4335-bd2d-181b801e41b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.701153 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4ca0a-86dd-4335-bd2d-181b801e41b6 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.710724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251137f-ee3e-46de-b211-1a03ef706f61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:48.711033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1295/2618] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:50:48.712146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251137f-ee3e-46de-b211-1a03ef706f61 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:48.721264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1324/2619] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:49.638017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4ca0a-86dd-4335-bd2d-181b801e41b6 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:49.648923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4ca0a-86dd-4335-bd2d-181b801e41b6 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:49.654768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d453818-e56c-4abb-8eb8-5eba180f4a0e tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7068d3ec9a4287b2dffbe2e8d40d47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e97f38fb983416181d5267b7d92acb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:49.655552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453818-e56c-4abb-8eb8-5eba180f4a0e tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:49.655843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453818-e56c-4abb-8eb8-5eba180f4a0e tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:49.656158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453818-e56c-4abb-8eb8-5eba180f4a0e tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:49.656989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d453818-e56c-4abb-8eb8-5eba180f4a0e tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:50:49.657575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d453818-e56c-4abb-8eb8-5eba180f4a0e tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:49.660158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d453818-e56c-4abb-8eb8-5eba180f4a0e tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:49.676451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1296/2620] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:50:49.680752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251137f-ee3e-46de-b211-1a03ef706f61 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:49.697680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251137f-ee3e-46de-b211-1a03ef706f61 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:49.700330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5337a-96ae-48f4-bf18-246d63583764 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7068d3ec9a4287b2dffbe2e8d40d47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e97f38fb983416181d5267b7d92acb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:49.700947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5337a-96ae-48f4-bf18-246d63583764 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:49.701226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5337a-96ae-48f4-bf18-246d63583764 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:49.701555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5337a-96ae-48f4-bf18-246d63583764 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:49.702219 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5337a-96ae-48f4-bf18-246d63583764 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:49.704570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5337a-96ae-48f4-bf18-246d63583764 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:49.798610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d453818-e56c-4abb-8eb8-5eba180f4a0e tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:49.801044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d453818-e56c-4abb-8eb8-5eba180f4a0e tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:49.817231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:49.853919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1325/2621] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:49 2022] POST /identity/v3/projects => generated 354 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:49.854996 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5337a-96ae-48f4-bf18-246d63583764 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:49.869858 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d453818-e56c-4abb-8eb8-5eba180f4a0e tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:49.896363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:49.929865 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5337a-96ae-48f4-bf18-246d63583764 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:49.934022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a647d2-dae4-4d79-95b7-acf018807266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:49.934789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a647d2-dae4-4d79-95b7-acf018807266 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:49.934926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2b60-ae14-43b6-a912-85b229f8feb0 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7068d3ec9a4287b2dffbe2e8d40d47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e97f38fb983416181d5267b7d92acb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:49.935194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a647d2-dae4-4d79-95b7-acf018807266 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:49.935483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2b60-ae14-43b6-a912-85b229f8feb0 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:49.935610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a647d2-dae4-4d79-95b7-acf018807266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:49.935763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2b60-ae14-43b6-a912-85b229f8feb0 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:49.936039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2b60-ae14-43b6-a912-85b229f8feb0 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:49.936706 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2b60-ae14-43b6-a912-85b229f8feb0 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:50:49.937027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2b60-ae14-43b6-a912-85b229f8feb0 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:49.937703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a7a647d2-dae4-4d79-95b7-acf018807266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:49.938665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2b60-ae14-43b6-a912-85b229f8feb0 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:49.952120 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1326/2622] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:50:50.006394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a647d2-dae4-4d79-95b7-acf018807266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:50.007897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a647d2-dae4-4d79-95b7-acf018807266 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:50.018150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1297/2623] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:50:50.461462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2b60-ae14-43b6-a912-85b229f8feb0 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:50.475550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2b60-ae14-43b6-a912-85b229f8feb0 tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:50.484086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da7068d3ec9a4287b2dffbe2e8d40d47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e97f38fb983416181d5267b7d92acb1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:50.485459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:50.485825 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:50.486201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] PATH_INFO: `/v3/projects/583101b61dac4e48a1816aacf3217de7` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:50.488282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: Authorizing `identity:delete_project(project_id=583101b61dac4e48a1816aacf3217de7)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:50.491803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:50.638903 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583101b61dac4e48a1816aacf3217de7', 'request_id': 'req-d2342e5a-cba1-4075-bf47-e329be3339ef'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:50.646870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583101b61dac4e48a1816aacf3217de7', 'request_id': 'req-d2342e5a-cba1-4075-bf47-e329be3339ef'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:50.729784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583101b61dac4e48a1816aacf3217de7', 'request_id': 'req-d2342e5a-cba1-4075-bf47-e329be3339ef'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:50.745318 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] The token cache is being invalidate because project 583101b61dac4e48a1816aacf3217de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:50.745613 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:50.746377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:50.748918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1327/2624] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:50 2022] DELETE /identity/v3/projects/583101b61dac4e48a1816aacf3217de7 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:51.079406 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a647d2-dae4-4d79-95b7-acf018807266 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:51.141462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a647d2-dae4-4d79-95b7-acf018807266 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:51.187046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:51.216447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667509c-ecf9-4451-bfad-a4461c114482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:51.219368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667509c-ecf9-4451-bfad-a4461c114482 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:51.219653 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667509c-ecf9-4451-bfad-a4461c114482 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:51.221239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667509c-ecf9-4451-bfad-a4461c114482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:51.226328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5667509c-ecf9-4451-bfad-a4461c114482 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:51.352230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667509c-ecf9-4451-bfad-a4461c114482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:51.354305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667509c-ecf9-4451-bfad-a4461c114482 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:51.366425 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1298/2625] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:51.747381 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:51.753360 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:51.891075 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2342e5a-cba1-4075-bf47-e329be3339ef tempest-VolumeQuotaClassesTest-799990971 tempest-VolumeQuotaClassesTest-799990971-project-admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:51.900771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:51.901676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:51.901884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:51.902212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] PATH_INFO: `/v3/users/42fce22f3a464a69a4c385540ad9e9d2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:51.903792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42fce22f3a464a69a4c385540ad9e9d2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:51.906560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:52.180199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:52.182008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42fce22f3a464a69a4c385540ad9e9d2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:52.210574 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42fce22f3a464a69a4c385540ad9e9d2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:52.212905 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42fce22f3a464a69a4c385540ad9e9d2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:52.309448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42fce22f3a464a69a4c385540ad9e9d2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:52.310236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42fce22f3a464a69a4c385540ad9e9d2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:52.331764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1328/2626] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:51 2022] DELETE /identity/v3/users/42fce22f3a464a69a4c385540ad9e9d2 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:52.333726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc812-f21f-49ad-9d4c-3f7aee308c5b None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:52.334711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc812-f21f-49ad-9d4c-3f7aee308c5b None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:52.335189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc812-f21f-49ad-9d4c-3f7aee308c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:52.363853 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-28afc812-f21f-49ad-9d4c-3f7aee308c5b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:52.365521 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-28afc812-f21f-49ad-9d4c-3f7aee308c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:52.456702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1299/2627] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:52.466284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:52.467913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:52.553860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f490840-cffc-40f9-a91f-cb6341235aa2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:52.558841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2af4e-5d2a-4208-b831-ac0fcf8cdd38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:52.559794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2af4e-5d2a-4208-b831-ac0fcf8cdd38 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:52.560462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2af4e-5d2a-4208-b831-ac0fcf8cdd38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:52.560840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2af4e-5d2a-4208-b831-ac0fcf8cdd38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:52.573989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2af4e-5d2a-4208-b831-ac0fcf8cdd38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:52.578015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2af4e-5d2a-4208-b831-ac0fcf8cdd38 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:52.605639 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1329/2628] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:53.071339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afc812-f21f-49ad-9d4c-3f7aee308c5b None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:53.072855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-28afc812-f21f-49ad-9d4c-3f7aee308c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:53.135614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afc812-f21f-49ad-9d4c-3f7aee308c5b None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:53.139753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebd080-87c1-45da-b2e2-db39d77a825a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:53.145533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebd080-87c1-45da-b2e2-db39d77a825a service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:53.145533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebd080-87c1-45da-b2e2-db39d77a825a service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:53.145533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebd080-87c1-45da-b2e2-db39d77a825a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:53.146279 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2af4e-5d2a-4208-b831-ac0fcf8cdd38 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:53.146706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-71ebd080-87c1-45da-b2e2-db39d77a825a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:53.146811 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d5e2af4e-5d2a-4208-b831-ac0fcf8cdd38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:53.159267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2af4e-5d2a-4208-b831-ac0fcf8cdd38 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:53.165197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:53.166061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:53.168175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:53.168175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] PATH_INFO: `/v3/projects/e59f33ee9d9a442688de65d7ee796701` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:53.168175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e59f33ee9d9a442688de65d7ee796701)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:53.170752 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:53.227149 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:53.248766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebd080-87c1-45da-b2e2-db39d77a825a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:53.250323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebd080-87c1-45da-b2e2-db39d77a825a service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:53.260186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1300/2629] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:53.307549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e59f33ee9d9a442688de65d7ee796701', 'request_id': 'req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:53.314185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e59f33ee9d9a442688de65d7ee796701', 'request_id': 'req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:53.396213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e59f33ee9d9a442688de65d7ee796701', 'request_id': 'req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:53.407344 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] The token cache is being invalidate because project e59f33ee9d9a442688de65d7ee796701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:53.407630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:53.408740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:53.410458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1330/2630] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:53 2022] DELETE /identity/v3/projects/e59f33ee9d9a442688de65d7ee796701 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:53.414231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebd080-87c1-45da-b2e2-db39d77a825a service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:53.425305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebd080-87c1-45da-b2e2-db39d77a825a service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:53.491348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:53.491808 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:53.492084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:53.492360 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] PATH_INFO: `/v3/users/da7068d3ec9a4287b2dffbe2e8d40d47` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:53.493536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da7068d3ec9a4287b2dffbe2e8d40d47)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:53.495493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:53.697668 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da7068d3ec9a4287b2dffbe2e8d40d47'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:53.698415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da7068d3ec9a4287b2dffbe2e8d40d47'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:53.712345 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da7068d3ec9a4287b2dffbe2e8d40d47'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:53.791031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da7068d3ec9a4287b2dffbe2e8d40d47'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:53.791649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da7068d3ec9a4287b2dffbe2e8d40d47'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:53.804380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1301/2631] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:53 2022] DELETE /identity/v3/users/da7068d3ec9a4287b2dffbe2e8d40d47 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:54.406095 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:54.407630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:54.421148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37cdf2-53e4-4ab9-93c4-4b5407138f76 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:54.474792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:54.488260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:54.488901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:54.489184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:54.489520 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] PATH_INFO: `/v3/projects/0e97f38fb983416181d5267b7d92acb1` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:54.490653 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e97f38fb983416181d5267b7d92acb1)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:54.492583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:54.602392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e97f38fb983416181d5267b7d92acb1', 'request_id': 'req-1f8397bd-e87c-426c-a8d2-6f5338877266'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:54.609078 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e97f38fb983416181d5267b7d92acb1', 'request_id': 'req-1f8397bd-e87c-426c-a8d2-6f5338877266'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:54.666738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:54.668309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:54.682961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e97f38fb983416181d5267b7d92acb1', 'request_id': 'req-1f8397bd-e87c-426c-a8d2-6f5338877266'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:54.683130 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2fd0f-8f05-406d-8aa2-33f19352c558 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:54.695330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] The token cache is being invalidate because project 0e97f38fb983416181d5267b7d92acb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:54.695625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:54.696443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-1f8397bd-e87c-426c-a8d2-6f5338877266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:54.698133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1331/2632] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:54 2022] DELETE /identity/v3/projects/0e97f38fb983416181d5267b7d92acb1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:54.702910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccadc40-c12e-4f4b-be37-d8386c6d79c4 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:54.703197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccadc40-c12e-4f4b-be37-d8386c6d79c4 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:54.703492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccadc40-c12e-4f4b-be37-d8386c6d79c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:54.723340 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2ccadc40-c12e-4f4b-be37-d8386c6d79c4 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:54.724553 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2ccadc40-c12e-4f4b-be37-d8386c6d79c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:54.737392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:54.743279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:54.743944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:54.744221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:54.744496 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] PATH_INFO: `/v3/users/6f1635c9c6cf4fb092c0f03a02e36174` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:54.745645 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f1635c9c6cf4fb092c0f03a02e36174)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:54.747585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:54.785902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1332/2633] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:54.794163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3585e0-7bef-47e0-b4f5-74ab899dc56d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:54.794465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3585e0-7bef-47e0-b4f5-74ab899dc56d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:54.794761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3585e0-7bef-47e0-b4f5-74ab899dc56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:54.814485 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-0e3585e0-7bef-47e0-b4f5-74ab899dc56d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:54.815681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0e3585e0-7bef-47e0-b4f5-74ab899dc56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:54.823900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1333/2634] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:54.827830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3585e0-7bef-47e0-b4f5-74ab899dc56d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:54.836883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3585e0-7bef-47e0-b4f5-74ab899dc56d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:54.841700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733a2b6-3459-44b3-b239-5f4429d484c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:54.842308 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a2b6-3459-44b3-b239-5f4429d484c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:54.842588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a2b6-3459-44b3-b239-5f4429d484c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:54.842860 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733a2b6-3459-44b3-b239-5f4429d484c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:54.843488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733a2b6-3459-44b3-b239-5f4429d484c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:50:54.843789 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733a2b6-3459-44b3-b239-5f4429d484c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:54.845391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733a2b6-3459-44b3-b239-5f4429d484c7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:54.857731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1334/2635] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:50:54.862582 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733a2b6-3459-44b3-b239-5f4429d484c7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:54.872005 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733a2b6-3459-44b3-b239-5f4429d484c7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:54.876741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c3351-0617-4598-a03a-82b11c7a0453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:54.878288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c3351-0617-4598-a03a-82b11c7a0453 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:54.878288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c3351-0617-4598-a03a-82b11c7a0453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:54.878288 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c3351-0617-4598-a03a-82b11c7a0453 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:54.878568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c3351-0617-4598-a03a-82b11c7a0453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:54.880420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c3351-0617-4598-a03a-82b11c7a0453 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:54.942018 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f1635c9c6cf4fb092c0f03a02e36174'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:54.942700 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f1635c9c6cf4fb092c0f03a02e36174'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:54.954733 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f1635c9c6cf4fb092c0f03a02e36174'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:55.010035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1335/2636] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:54 2022] POST /identity/v3/projects => generated 391 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:55.013977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c3351-0617-4598-a03a-82b11c7a0453 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:55.015594 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-091c3351-0617-4598-a03a-82b11c7a0453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:55.031269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f1635c9c6cf4fb092c0f03a02e36174'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:55.031957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f1635c9c6cf4fb092c0f03a02e36174'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:55.045378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1302/2637] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:50:54 2022] DELETE /identity/v3/users/6f1635c9c6cf4fb092c0f03a02e36174 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:55.061359 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:55.085365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c3351-0617-4598-a03a-82b11c7a0453 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:55.141956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1ef0-e144-4967-b18b-982e49e62f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:55.143115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1ef0-e144-4967-b18b-982e49e62f1d admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.143393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1ef0-e144-4967-b18b-982e49e62f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.143665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1ef0-e144-4967-b18b-982e49e62f1d admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.144302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df1ef0-e144-4967-b18b-982e49e62f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:55.146306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df1ef0-e144-4967-b18b-982e49e62f1d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:55.282940 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1336/2638] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:55 2022] POST /identity/v3/users => generated 296 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:55.287584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:55.297717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be3db8-acb2-4579-9a03-ac8a2f948e11 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:55.302460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba886825-2440-4d47-8e05-46ace38f4085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:55.308748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba886825-2440-4d47-8e05-46ace38f4085 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.308748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba886825-2440-4d47-8e05-46ace38f4085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.308748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba886825-2440-4d47-8e05-46ace38f4085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.308748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba886825-2440-4d47-8e05-46ace38f4085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:55.308748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba886825-2440-4d47-8e05-46ace38f4085 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:55.314979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1ef0-e144-4967-b18b-982e49e62f1d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:55.317268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e1df1ef0-e144-4967-b18b-982e49e62f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:55.323994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1303/2639] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:55.328037 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba886825-2440-4d47-8e05-46ace38f4085 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:55.333391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1ef0-e144-4967-b18b-982e49e62f1d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:55.339493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:55.340236 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.340520 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba886825-2440-4d47-8e05-46ace38f4085 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:55.340630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.341410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] PATH_INFO: `/v3/projects/636e6b38bf1d4736a38728488881e9d2` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.342129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=636e6b38bf1d4736a38728488881e9d2)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:55.344354 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:55.354594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b34da-cf57-4f25-92a6-7b55dc3f3949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:55.355216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b34da-cf57-4f25-92a6-7b55dc3f3949 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.355575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b34da-cf57-4f25-92a6-7b55dc3f3949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.355855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b34da-cf57-4f25-92a6-7b55dc3f3949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.356603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b34da-cf57-4f25-92a6-7b55dc3f3949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:55.358339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b34da-cf57-4f25-92a6-7b55dc3f3949 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:55.369845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1304/2640] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:55.373569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b34da-cf57-4f25-92a6-7b55dc3f3949 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:55.384182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b34da-cf57-4f25-92a6-7b55dc3f3949 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:55.388900 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a31f1-6b04-443b-beb3-ab0161f3c954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:55.389554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a31f1-6b04-443b-beb3-ab0161f3c954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.389838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a31f1-6b04-443b-beb3-ab0161f3c954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.390111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a31f1-6b04-443b-beb3-ab0161f3c954 admin admin] PATH_INFO: `/v3/projects/775d4a7723de41c89a362a1460cc5f2f/users/5ddc51f1b75c4fc198818e60e0030818/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.405702 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a31f1-6b04-443b-beb3-ab0161f3c954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=775d4a7723de41c89a362a1460cc5f2f, user_id=5ddc51f1b75c4fc198818e60e0030818, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:55.409176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a31f1-6b04-443b-beb3-ab0161f3c954 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:55.523473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1305/2641] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:55 2022] PUT /identity/v3/projects/775d4a7723de41c89a362a1460cc5f2f/users/5ddc51f1b75c4fc198818e60e0030818/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:55.532504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636e6b38bf1d4736a38728488881e9d2', 'request_id': 'req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:55.533293 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9c456-0035-403f-a24b-08f9c89b38e7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.533527 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9c456-0035-403f-a24b-08f9c89b38e7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.533801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9c456-0035-403f-a24b-08f9c89b38e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.540050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636e6b38bf1d4736a38728488881e9d2', 'request_id': 'req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:55.562552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4bd9c456-0035-403f-a24b-08f9c89b38e7 None None] MFA Rules not processed for user `5ddc51f1b75c4fc198818e60e0030818`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:55.563764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4bd9c456-0035-403f-a24b-08f9c89b38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:55.638743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636e6b38bf1d4736a38728488881e9d2', 'request_id': 'req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:55.643451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1306/2642] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:55.646500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9c456-0035-403f-a24b-08f9c89b38e7 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:55.654970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] The token cache is being invalidate because project 636e6b38bf1d4736a38728488881e9d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:50:55.655163 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:55.655763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5d73ff2a-4f5d-41e1-9e3f-4363fe6d12c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:50:55.659425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1337/2643] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:50:55 2022] DELETE /identity/v3/projects/636e6b38bf1d4736a38728488881e9d2 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:55.663388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90506236-c4b8-410d-b45f-327aacf5c17a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.663627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90506236-c4b8-410d-b45f-327aacf5c17a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.664020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90506236-c4b8-410d-b45f-327aacf5c17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.675546 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:55.687193 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-90506236-c4b8-410d-b45f-327aacf5c17a None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:55.688535 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-90506236-c4b8-410d-b45f-327aacf5c17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:55.723611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9c456-0035-403f-a24b-08f9c89b38e7 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:55.728716 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c62b7-2a80-4c7e-898d-5a39d5edd496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:55.729322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c62b7-2a80-4c7e-898d-5a39d5edd496 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.729687 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c62b7-2a80-4c7e-898d-5a39d5edd496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.729872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c62b7-2a80-4c7e-898d-5a39d5edd496 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.730962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c62b7-2a80-4c7e-898d-5a39d5edd496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:55.732434 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c62b7-2a80-4c7e-898d-5a39d5edd496 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:55.750533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:55.760197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1338/2644] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:50:55.768302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c0584-951e-4b6c-a591-6fd10e97df4a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.768586 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c0584-951e-4b6c-a591-6fd10e97df4a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.768855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c0584-951e-4b6c-a591-6fd10e97df4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.788874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-761c0584-951e-4b6c-a591-6fd10e97df4a None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:55.789112 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-761c0584-951e-4b6c-a591-6fd10e97df4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:55.796771 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1339/2645] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:55.800711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c0584-951e-4b6c-a591-6fd10e97df4a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:55.809188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c0584-951e-4b6c-a591-6fd10e97df4a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:55.813657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad856dba-7fb6-4788-9ad5-e0bc1e3ebe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:55.814747 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad856dba-7fb6-4788-9ad5-e0bc1e3ebe59 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.817589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad856dba-7fb6-4788-9ad5-e0bc1e3ebe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.817589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad856dba-7fb6-4788-9ad5-e0bc1e3ebe59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.817589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad856dba-7fb6-4788-9ad5-e0bc1e3ebe59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:50:55.817589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad856dba-7fb6-4788-9ad5-e0bc1e3ebe59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:55.817999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad856dba-7fb6-4788-9ad5-e0bc1e3ebe59 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:55.829363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1340/2646] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:50:55.842184 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad856dba-7fb6-4788-9ad5-e0bc1e3ebe59 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:55.847913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1307/2647] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:55 2022] POST /identity/v3/projects => generated 391 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:55.848839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c62b7-2a80-4c7e-898d-5a39d5edd496 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:55.850501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-404c62b7-2a80-4c7e-898d-5a39d5edd496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:55.852178 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad856dba-7fb6-4788-9ad5-e0bc1e3ebe59 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:55.912428 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167758b-f36d-49aa-89ba-09d35773ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:55.915361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167758b-f36d-49aa-89ba-09d35773ac0c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.915361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167758b-f36d-49aa-89ba-09d35773ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.915361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167758b-f36d-49aa-89ba-09d35773ac0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.915361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167758b-f36d-49aa-89ba-09d35773ac0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:55.916141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167758b-f36d-49aa-89ba-09d35773ac0c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:55.919020 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c62b7-2a80-4c7e-898d-5a39d5edd496 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:55.924750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94fbb-90a1-421a-ac07-3e3b94b0043d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:55.925253 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94fbb-90a1-421a-ac07-3e3b94b0043d admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:55.925535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94fbb-90a1-421a-ac07-3e3b94b0043d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:55.925790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94fbb-90a1-421a-ac07-3e3b94b0043d admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:55.926438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba94fbb-90a1-421a-ac07-3e3b94b0043d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:55.928335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba94fbb-90a1-421a-ac07-3e3b94b0043d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:55.995857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1308/2648] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:55 2022] POST /identity/v3/users => generated 302 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:55.996066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1341/2649] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:50:55 2022] POST /identity/v3/projects => generated 369 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:50:55.997290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167758b-f36d-49aa-89ba-09d35773ac0c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:55.997422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94fbb-90a1-421a-ac07-3e3b94b0043d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:56.009039 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94fbb-90a1-421a-ac07-3e3b94b0043d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:56.058672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167758b-f36d-49aa-89ba-09d35773ac0c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:56.063424 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444101-2310-4279-ba70-46817a2d56c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:56.064073 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444101-2310-4279-ba70-46817a2d56c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.065031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444101-2310-4279-ba70-46817a2d56c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.065307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444101-2310-4279-ba70-46817a2d56c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.065933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2444101-2310-4279-ba70-46817a2d56c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:56.068215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2444101-2310-4279-ba70-46817a2d56c9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:56.068344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8163-8254-4902-9b4c-522996f98543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:56.068946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8163-8254-4902-9b4c-522996f98543 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.069222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8163-8254-4902-9b4c-522996f98543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.069488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8163-8254-4902-9b4c-522996f98543 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.070184 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a8163-8254-4902-9b4c-522996f98543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:56.071787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a8163-8254-4902-9b4c-522996f98543 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:56.082228 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1309/2650] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:56.086122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8163-8254-4902-9b4c-522996f98543 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:56.095837 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8163-8254-4902-9b4c-522996f98543 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:56.100536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1950d-c581-47de-8904-f2a016cf6a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:56.101161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1950d-c581-47de-8904-f2a016cf6a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.107734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1950d-c581-47de-8904-f2a016cf6a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.107734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1950d-c581-47de-8904-f2a016cf6a65 admin admin] PATH_INFO: `/v3/projects/1392d5333d80426588ff5c5c189e2c38/users/687847b45aba44c8838ab953cdcabe69/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.116584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1950d-c581-47de-8904-f2a016cf6a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1392d5333d80426588ff5c5c189e2c38, user_id=687847b45aba44c8838ab953cdcabe69, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:56.119285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1950d-c581-47de-8904-f2a016cf6a65 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:56.145532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1342/2651] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:50:55 2022] POST /identity/v3/users => generated 285 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:50:56.148966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444101-2310-4279-ba70-46817a2d56c9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:56.158727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444101-2310-4279-ba70-46817a2d56c9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:56.163370 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd858e-ab7f-46f1-8c3c-5c5af70306bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:56.165133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd858e-ab7f-46f1-8c3c-5c5af70306bd admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.165133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd858e-ab7f-46f1-8c3c-5c5af70306bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.165133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd858e-ab7f-46f1-8c3c-5c5af70306bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.165133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd858e-ab7f-46f1-8c3c-5c5af70306bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:56.166738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd858e-ab7f-46f1-8c3c-5c5af70306bd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:56.177053 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1343/2652] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:56.181165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd858e-ab7f-46f1-8c3c-5c5af70306bd admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:56.185962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1310/2653] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:56 2022] PUT /identity/v3/projects/1392d5333d80426588ff5c5c189e2c38/users/687847b45aba44c8838ab953cdcabe69/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:56.190356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1950d-c581-47de-8904-f2a016cf6a65 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:56.195408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd858e-ab7f-46f1-8c3c-5c5af70306bd admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:56.199619 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:56.238391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:56.254819 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f838218-4ad8-4443-ae03-3d2a94cdd77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:56.255457 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f838218-4ad8-4443-ae03-3d2a94cdd77d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.255731 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f838218-4ad8-4443-ae03-3d2a94cdd77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.256000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f838218-4ad8-4443-ae03-3d2a94cdd77d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.256634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f838218-4ad8-4443-ae03-3d2a94cdd77d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:56.258055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1950d-c581-47de-8904-f2a016cf6a65 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:56.258279 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f838218-4ad8-4443-ae03-3d2a94cdd77d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:56.263036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7350274-4738-4eba-82c8-a823007680b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:56.263698 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350274-4738-4eba-82c8-a823007680b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.263981 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350274-4738-4eba-82c8-a823007680b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.264258 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350274-4738-4eba-82c8-a823007680b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.264958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7350274-4738-4eba-82c8-a823007680b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:56.266564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7350274-4738-4eba-82c8-a823007680b2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:56.268911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1344/2654] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:50:56.272797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f838218-4ad8-4443-ae03-3d2a94cdd77d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:56.281462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1311/2655] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:50:56.282517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7350274-4738-4eba-82c8-a823007680b2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:56.283907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f838218-4ad8-4443-ae03-3d2a94cdd77d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:56.288688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7fce2-0946-41b9-ae7f-80286845ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:56.291459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7fce2-0946-41b9-ae7f-80286845ecf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.291459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7fce2-0946-41b9-ae7f-80286845ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.291459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7fce2-0946-41b9-ae7f-80286845ecf0 admin admin] PATH_INFO: `/v3/projects/a3a0e8fcefcf4cd5bdb012232d95f3ce/users/724037b049ec4714a148fcd977463231/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.292776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7350274-4738-4eba-82c8-a823007680b2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:56.297501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b7a36-f512-4102-ad1e-fbe1a6f28ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:56.298242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7a36-f512-4102-ad1e-fbe1a6f28ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.298485 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7a36-f512-4102-ad1e-fbe1a6f28ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.299307 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7a36-f512-4102-ad1e-fbe1a6f28ce6 admin admin] PATH_INFO: `/v3/projects/1392d5333d80426588ff5c5c189e2c38/users/687847b45aba44c8838ab953cdcabe69/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.301240 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b7a36-f512-4102-ad1e-fbe1a6f28ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1392d5333d80426588ff5c5c189e2c38, user_id=687847b45aba44c8838ab953cdcabe69, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:56.303909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b7a36-f512-4102-ad1e-fbe1a6f28ce6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:56.306827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7fce2-0946-41b9-ae7f-80286845ecf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3a0e8fcefcf4cd5bdb012232d95f3ce, user_id=724037b049ec4714a148fcd977463231, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:56.309665 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7fce2-0946-41b9-ae7f-80286845ecf0 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:56.314533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1312/2656] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:56 2022] PUT /identity/v3/projects/1392d5333d80426588ff5c5c189e2c38/users/687847b45aba44c8838ab953cdcabe69/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:50:56.325780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2394c6-0991-486e-94e8-86f1c2a0a1aa None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.326065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2394c6-0991-486e-94e8-86f1c2a0a1aa None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.326396 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2394c6-0991-486e-94e8-86f1c2a0a1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.353917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9d2394c6-0991-486e-94e8-86f1c2a0a1aa None None] MFA Rules not processed for user `687847b45aba44c8838ab953cdcabe69`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:56.355163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d2394c6-0991-486e-94e8-86f1c2a0a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:56.397071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1345/2657] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:50:56 2022] PUT /identity/v3/projects/a3a0e8fcefcf4cd5bdb012232d95f3ce/users/724037b049ec4714a148fcd977463231/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:50:56.409539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb6dfd-0f98-41aa-b69b-c5d76f89cdde None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.409993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb6dfd-0f98-41aa-b69b-c5d76f89cdde None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.410305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb6dfd-0f98-41aa-b69b-c5d76f89cdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.431413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1313/2658] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:56 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:56.440655 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-23fb6dfd-0f98-41aa-b69b-c5d76f89cdde None None] MFA Rules not processed for user `724037b049ec4714a148fcd977463231`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:50:56.442165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-23fb6dfd-0f98-41aa-b69b-c5d76f89cdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:56.442931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2394c6-0991-486e-94e8-86f1c2a0a1aa None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:56.444451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9d2394c6-0991-486e-94e8-86f1c2a0a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:56.511229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2394c6-0991-486e-94e8-86f1c2a0a1aa None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:56.523766 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1346/2659] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:50:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:50:56.537400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb6dfd-0f98-41aa-b69b-c5d76f89cdde None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:56.539335 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-23fb6dfd-0f98-41aa-b69b-c5d76f89cdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:56.544447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:50:56.571182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55947c96-52c2-44b6-be4c-39eea14c498b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:56.571558 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55947c96-52c2-44b6-be4c-39eea14c498b service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.571844 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55947c96-52c2-44b6-be4c-39eea14c498b service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.572118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55947c96-52c2-44b6-be4c-39eea14c498b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.653443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55947c96-52c2-44b6-be4c-39eea14c498b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:56.654958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55947c96-52c2-44b6-be4c-39eea14c498b service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:56.658823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb6dfd-0f98-41aa-b69b-c5d76f89cdde None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:56.665190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e57b8c-43a0-4073-ac21-19bb8ea644c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:56.665465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1314/2660] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:56 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:50:56.665575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e57b8c-43a0-4073-ac21-19bb8ea644c8 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:56.665840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e57b8c-43a0-4073-ac21-19bb8ea644c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:56.666166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e57b8c-43a0-4073-ac21-19bb8ea644c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:56.675591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e57b8c-43a0-4073-ac21-19bb8ea644c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:56.677646 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e57b8c-43a0-4073-ac21-19bb8ea644c8 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:56.689972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1347/2661] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:50:57.206023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55947c96-52c2-44b6-be4c-39eea14c498b service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:50:57.208451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-55947c96-52c2-44b6-be4c-39eea14c498b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:57.220071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55947c96-52c2-44b6-be4c-39eea14c498b service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:50:57.223643 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438b6a5-15fc-4723-a730-9c3c921365cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:50:57.224237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438b6a5-15fc-4723-a730-9c3c921365cb service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:50:57.224511 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438b6a5-15fc-4723-a730-9c3c921365cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:50:57.224781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438b6a5-15fc-4723-a730-9c3c921365cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:50:57.226465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c438b6a5-15fc-4723-a730-9c3c921365cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:50:57.289071 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438b6a5-15fc-4723-a730-9c3c921365cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:50:57.290549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438b6a5-15fc-4723-a730-9c3c921365cb service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:50:57.310172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1315/2662] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:50:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:00.131762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e57b8c-43a0-4073-ac21-19bb8ea644c8 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:00.138336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:00.210853 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e57b8c-43a0-4073-ac21-19bb8ea644c8 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:00.218889 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:00.219537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:00.219814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:00.220068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] PATH_INFO: `/v3/users/724037b049ec4714a148fcd977463231` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:00.221257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] RBAC: Authorizing `identity:delete_user(user_id=724037b049ec4714a148fcd977463231)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:00.223210 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:00.237976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438b6a5-15fc-4723-a730-9c3c921365cb service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:00.241804 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c438b6a5-15fc-4723-a730-9c3c921365cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:00.256678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438b6a5-15fc-4723-a730-9c3c921365cb service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:00.261720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:00.264448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:00.264448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:00.264448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] PATH_INFO: `/v3/users/132dd0a1b18d4083a5aa0bb174d383f7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:00.264448 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=132dd0a1b18d4083a5aa0bb174d383f7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:00.265984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:00.371679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724037b049ec4714a148fcd977463231'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:00.384148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724037b049ec4714a148fcd977463231'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:00.384813 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724037b049ec4714a148fcd977463231'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:00.421966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '132dd0a1b18d4083a5aa0bb174d383f7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:00.422782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '132dd0a1b18d4083a5aa0bb174d383f7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:00.436598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '132dd0a1b18d4083a5aa0bb174d383f7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:00.443496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724037b049ec4714a148fcd977463231'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:00.444496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-788b0bf0-c539-484b-a50d-80e73bfe732c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724037b049ec4714a148fcd977463231'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:00.460681 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1348/2663] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:00 2022] DELETE /identity/v3/users/724037b049ec4714a148fcd977463231 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:00.462305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5aa20-5a51-46d0-ac65-447301b2eaf5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:00.465488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5aa20-5a51-46d0-ac65-447301b2eaf5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:00.465488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5aa20-5a51-46d0-ac65-447301b2eaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:00.484202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-58b5aa20-5a51-46d0-ac65-447301b2eaf5 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:00.485598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-58b5aa20-5a51-46d0-ac65-447301b2eaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:00.488215 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '132dd0a1b18d4083a5aa0bb174d383f7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:00.490123 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-638b35e5-9ebf-4707-a7ff-b01b54c366ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '132dd0a1b18d4083a5aa0bb174d383f7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:00.493607 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:00.507694 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1316/2664] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:00 2022] DELETE /identity/v3/users/132dd0a1b18d4083a5aa0bb174d383f7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:00.517141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864150f-f99c-49d8-8754-f397df416059 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:00.517141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864150f-f99c-49d8-8754-f397df416059 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:00.517141 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864150f-f99c-49d8-8754-f397df416059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:00.537471 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-6864150f-f99c-49d8-8754-f397df416059 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:00.538723 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6864150f-f99c-49d8-8754-f397df416059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:00.563484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1349/2665] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:00.572777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5aa20-5a51-46d0-ac65-447301b2eaf5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:00.574224 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-58b5aa20-5a51-46d0-ac65-447301b2eaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:00.598536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1317/2666] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:00.600798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:00.608871 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864150f-f99c-49d8-8754-f397df416059 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:00.610526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6864150f-f99c-49d8-8754-f397df416059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:00.661051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5aa20-5a51-46d0-ac65-447301b2eaf5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:00.665519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a20081-6d4a-4000-b7c2-f51a65c78106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:00.666231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a20081-6d4a-4000-b7c2-f51a65c78106 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:00.666447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a20081-6d4a-4000-b7c2-f51a65c78106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:00.666713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a20081-6d4a-4000-b7c2-f51a65c78106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:00.668447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864150f-f99c-49d8-8754-f397df416059 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:00.672524 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad50df-6865-42f3-8f9a-7588f888711f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:00.673049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad50df-6865-42f3-8f9a-7588f888711f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:00.673327 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad50df-6865-42f3-8f9a-7588f888711f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:00.673598 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad50df-6865-42f3-8f9a-7588f888711f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:00.676407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a20081-6d4a-4000-b7c2-f51a65c78106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:00.678395 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a20081-6d4a-4000-b7c2-f51a65c78106 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:00.683249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad50df-6865-42f3-8f9a-7588f888711f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:00.684655 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad50df-6865-42f3-8f9a-7588f888711f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:00.688136 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1350/2667] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:00.694554 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1318/2668] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:51:01.370295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a20081-6d4a-4000-b7c2-f51a65c78106 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:01.371770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-74a20081-6d4a-4000-b7c2-f51a65c78106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:01.386280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad50df-6865-42f3-8f9a-7588f888711f service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:01.386280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7aad50df-6865-42f3-8f9a-7588f888711f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:01.387516 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a20081-6d4a-4000-b7c2-f51a65c78106 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:01.392741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:01.393703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:01.393988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:01.394565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] PATH_INFO: `/v3/projects/a3a0e8fcefcf4cd5bdb012232d95f3ce` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:01.395532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3a0e8fcefcf4cd5bdb012232d95f3ce)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:01.397476 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:01.411896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad50df-6865-42f3-8f9a-7588f888711f service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:01.425438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:01.426242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:01.426450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:01.426727 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] PATH_INFO: `/v3/projects/c286b5514e7f4da4a6a9377b7cc257c4` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:01.430774 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c286b5514e7f4da4a6a9377b7cc257c4)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:01.432739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:01.551757 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3a0e8fcefcf4cd5bdb012232d95f3ce', 'request_id': 'req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:01.559204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3a0e8fcefcf4cd5bdb012232d95f3ce', 'request_id': 'req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:01.592534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c286b5514e7f4da4a6a9377b7cc257c4', 'request_id': 'req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:01.599549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c286b5514e7f4da4a6a9377b7cc257c4', 'request_id': 'req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:01.636877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3a0e8fcefcf4cd5bdb012232d95f3ce', 'request_id': 'req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:01.647946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] The token cache is being invalidate because project a3a0e8fcefcf4cd5bdb012232d95f3ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:01.648306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:01.648934 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-9ee4e4ed-2425-4cd4-9520-6bc1132f9167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:01.650756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1351/2669] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:01 2022] DELETE /identity/v3/projects/a3a0e8fcefcf4cd5bdb012232d95f3ce => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:01.655157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3b728-ad16-480e-bc15-f2dcd295f255 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:01.655431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3b728-ad16-480e-bc15-f2dcd295f255 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:01.655698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3b728-ad16-480e-bc15-f2dcd295f255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:01.678727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-39c3b728-ad16-480e-bc15-f2dcd295f255 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:01.678727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-39c3b728-ad16-480e-bc15-f2dcd295f255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:01.729236 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c286b5514e7f4da4a6a9377b7cc257c4', 'request_id': 'req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:01.745255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] The token cache is being invalidate because project c286b5514e7f4da4a6a9377b7cc257c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:01.745806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:01.746748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:01.751683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1319/2670] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:01 2022] DELETE /identity/v3/projects/c286b5514e7f4da4a6a9377b7cc257c4 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:01.753265 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:01.754861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:01.767385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1352/2671] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:51:01.774759 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94308319-bfdb-4d16-b716-fda9f26827f7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:01.775119 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94308319-bfdb-4d16-b716-fda9f26827f7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:01.775464 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94308319-bfdb-4d16-b716-fda9f26827f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:01.776753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72935f7-de0a-446a-80b5-ab9e1ecd7ba0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:01.788683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:01.789311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:01.790398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:01.790686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] PATH_INFO: `/v3/users/54f5ddeac6d347259b4ecc5efb2d730f` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:01.791958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=54f5ddeac6d347259b4ecc5efb2d730f)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:01.794158 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:01.796182 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-94308319-bfdb-4d16-b716-fda9f26827f7 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:01.799484 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-94308319-bfdb-4d16-b716-fda9f26827f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:01.805617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1353/2672] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:51:01.808385 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:01.809991 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94308319-bfdb-4d16-b716-fda9f26827f7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:01.819770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94308319-bfdb-4d16-b716-fda9f26827f7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:01.824816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbe5a0-9a2e-4253-a550-cd0416639e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:01.830317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbe5a0-9a2e-4253-a550-cd0416639e9b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:01.830317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbe5a0-9a2e-4253-a550-cd0416639e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:01.830317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbe5a0-9a2e-4253-a550-cd0416639e9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:01.830317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffbe5a0-9a2e-4253-a550-cd0416639e9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:51:01.830317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffbe5a0-9a2e-4253-a550-cd0416639e9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:01.830317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffbe5a0-9a2e-4253-a550-cd0416639e9b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:01.840658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1354/2673] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:51:01.844962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbe5a0-9a2e-4253-a550-cd0416639e9b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:01.855465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbe5a0-9a2e-4253-a550-cd0416639e9b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:01.860271 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c17fe-f9f2-4591-a171-9b06506c0124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:01.860836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c17fe-f9f2-4591-a171-9b06506c0124 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:01.861108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c17fe-f9f2-4591-a171-9b06506c0124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:01.861402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c17fe-f9f2-4591-a171-9b06506c0124 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:01.862037 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c17fe-f9f2-4591-a171-9b06506c0124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:01.864036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c17fe-f9f2-4591-a171-9b06506c0124 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:01.870000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:02.058098 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1355/2674] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:01 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:51:02.062386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c17fe-f9f2-4591-a171-9b06506c0124 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:02.064516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54f5ddeac6d347259b4ecc5efb2d730f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:02.065244 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54f5ddeac6d347259b4ecc5efb2d730f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:02.078458 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54f5ddeac6d347259b4ecc5efb2d730f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:02.125262 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c17fe-f9f2-4591-a171-9b06506c0124 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:02.130237 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f29ef0-26e4-487a-a1e6-ee04f78f2596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:02.130846 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f29ef0-26e4-487a-a1e6-ee04f78f2596 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:02.131129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f29ef0-26e4-487a-a1e6-ee04f78f2596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:02.131408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f29ef0-26e4-487a-a1e6-ee04f78f2596 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:02.132093 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f29ef0-26e4-487a-a1e6-ee04f78f2596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:02.134019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f29ef0-26e4-487a-a1e6-ee04f78f2596 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:02.185591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54f5ddeac6d347259b4ecc5efb2d730f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:02.186304 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54f5ddeac6d347259b4ecc5efb2d730f'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:02.199914 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1320/2675] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:01 2022] DELETE /identity/v3/users/54f5ddeac6d347259b4ecc5efb2d730f => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:02.220550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:02.221958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:02.233105 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1356/2676] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:02 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:51:02.241856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f29ef0-26e4-487a-a1e6-ee04f78f2596 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:02.243675 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-01f29ef0-26e4-487a-a1e6-ee04f78f2596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:02.256674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f29ef0-26e4-487a-a1e6-ee04f78f2596 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:02.299742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b08ac-af15-498b-8d9c-a8ed7ee3b9fc admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:02.306235 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:02.306879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:02.307161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:02.307437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] PATH_INFO: `/v3/users/5ddc51f1b75c4fc198818e60e0030818` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:02.308744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddc51f1b75c4fc198818e60e0030818)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:02.310690 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:02.323985 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90e7a8-a0f3-44a9-924e-820740883bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:02.324622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90e7a8-a0f3-44a9-924e-820740883bcc admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:02.324919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90e7a8-a0f3-44a9-924e-820740883bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:02.325177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90e7a8-a0f3-44a9-924e-820740883bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:02.325828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90e7a8-a0f3-44a9-924e-820740883bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:02.327486 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90e7a8-a0f3-44a9-924e-820740883bcc admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:02.337351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1357/2677] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:51:02.341418 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90e7a8-a0f3-44a9-924e-820740883bcc admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:02.351454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90e7a8-a0f3-44a9-924e-820740883bcc admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:02.356026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c67644-f9df-4ddd-aed4-b8be9da54103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:02.356589 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67644-f9df-4ddd-aed4-b8be9da54103 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:02.356867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67644-f9df-4ddd-aed4-b8be9da54103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:02.357141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67644-f9df-4ddd-aed4-b8be9da54103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:02.357758 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c67644-f9df-4ddd-aed4-b8be9da54103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:02.359333 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c67644-f9df-4ddd-aed4-b8be9da54103 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:02.369727 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1358/2678] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:51:02.373697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c67644-f9df-4ddd-aed4-b8be9da54103 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:02.384052 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c67644-f9df-4ddd-aed4-b8be9da54103 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:02.389020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19476c-7317-4f7d-96bd-c0cdcb82ea53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:02.390307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19476c-7317-4f7d-96bd-c0cdcb82ea53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:02.390585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19476c-7317-4f7d-96bd-c0cdcb82ea53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:02.390855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19476c-7317-4f7d-96bd-c0cdcb82ea53 admin admin] PATH_INFO: `/v3/projects/fb5195738e9a49da8a1dcd11426e2d4c/users/4b01aa2e35204ae9a3e30d98abf870f3/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:02.409906 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19476c-7317-4f7d-96bd-c0cdcb82ea53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5195738e9a49da8a1dcd11426e2d4c, user_id=4b01aa2e35204ae9a3e30d98abf870f3, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:02.417570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19476c-7317-4f7d-96bd-c0cdcb82ea53 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:02.497250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1359/2679] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:02 2022] PUT /identity/v3/projects/fb5195738e9a49da8a1dcd11426e2d4c/users/4b01aa2e35204ae9a3e30d98abf870f3/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:02.508214 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4ccc0-5b1b-4dbf-b396-1883fb7f37b7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:02.508577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4ccc0-5b1b-4dbf-b396-1883fb7f37b7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:02.508920 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4ccc0-5b1b-4dbf-b396-1883fb7f37b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:02.514107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddc51f1b75c4fc198818e60e0030818'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:02.515344 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddc51f1b75c4fc198818e60e0030818'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:02.529776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddc51f1b75c4fc198818e60e0030818'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:02.540708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c9d4ccc0-5b1b-4dbf-b396-1883fb7f37b7 None None] MFA Rules not processed for user `4b01aa2e35204ae9a3e30d98abf870f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:02.541981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c9d4ccc0-5b1b-4dbf-b396-1883fb7f37b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:02.597133 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddc51f1b75c4fc198818e60e0030818'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:02.597904 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddc51f1b75c4fc198818e60e0030818'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:02.613592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1321/2680] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:02 2022] DELETE /identity/v3/users/5ddc51f1b75c4fc198818e60e0030818 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:02.614180 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1360/2681] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:51:02.618684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b1707-220c-4399-b697-b4450491a037 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:02.619307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b1707-220c-4399-b697-b4450491a037 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:02.619843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b1707-220c-4399-b697-b4450491a037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:02.623897 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:02.629297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:02.629297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:02.639408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:02.643448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7b8b1707-220c-4399-b697-b4450491a037 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:02.644703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7b8b1707-220c-4399-b697-b4450491a037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:02.716417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d19308-de93-4ac1-9e69-29d75bd30be2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:02.726784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022ba81-f668-457e-b2eb-efb539492247 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:02.727680 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022ba81-f668-457e-b2eb-efb539492247 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:02.728150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022ba81-f668-457e-b2eb-efb539492247 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:02.728584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022ba81-f668-457e-b2eb-efb539492247 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:02.740343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1361/2682] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:51:02.751756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b1707-220c-4399-b697-b4450491a037 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:02.753351 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7b8b1707-220c-4399-b697-b4450491a037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:02.807400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022ba81-f668-457e-b2eb-efb539492247 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:02.809859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022ba81-f668-457e-b2eb-efb539492247 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:02.823480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1322/2683] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:02.824488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022ba81-f668-457e-b2eb-efb539492247 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:02.825855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8022ba81-f668-457e-b2eb-efb539492247 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:02.841120 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022ba81-f668-457e-b2eb-efb539492247 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:02.859826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:02.860111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b1707-220c-4399-b697-b4450491a037 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:02.860488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:02.860776 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:02.861050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] PATH_INFO: `/v3/users/b0b9f106591a417fbbfe98947dbeae4c` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:02.862863 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0b9f106591a417fbbfe98947dbeae4c)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:02.864918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:02.867930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed52087-4e81-44e0-bd35-8518f480c7c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:02.867930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed52087-4e81-44e0-bd35-8518f480c7c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:02.870127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed52087-4e81-44e0-bd35-8518f480c7c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:02.870127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed52087-4e81-44e0-bd35-8518f480c7c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:02.881151 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed52087-4e81-44e0-bd35-8518f480c7c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:02.882818 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed52087-4e81-44e0-bd35-8518f480c7c5 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:02.894914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1362/2684] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:03.006788 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed52087-4e81-44e0-bd35-8518f480c7c5 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:03.008230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8ed52087-4e81-44e0-bd35-8518f480c7c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:03.030188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed52087-4e81-44e0-bd35-8518f480c7c5 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:03.046226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:03.046226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:03.046226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:03.046226 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] PATH_INFO: `/v3/projects/7ea33ee304ac4545898f04bc988da18c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:03.048471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea33ee304ac4545898f04bc988da18c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:03.048471 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:03.067758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0b9f106591a417fbbfe98947dbeae4c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.068613 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0b9f106591a417fbbfe98947dbeae4c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.084817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0b9f106591a417fbbfe98947dbeae4c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.157526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea33ee304ac4545898f04bc988da18c', 'request_id': 'req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.164124 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0b9f106591a417fbbfe98947dbeae4c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.165377 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea33ee304ac4545898f04bc988da18c', 'request_id': 'req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.165488 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0b9f106591a417fbbfe98947dbeae4c'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.179913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1323/2685] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:02 2022] DELETE /identity/v3/users/b0b9f106591a417fbbfe98947dbeae4c => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:03.180766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:03.182332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:03.203464 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:03.248284 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea33ee304ac4545898f04bc988da18c', 'request_id': 'req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.254640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:03.261754 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea133f-09ff-46ed-b31f-52ba761d64a2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:03.264383 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] The token cache is being invalidate because project 7ea33ee304ac4545898f04bc988da18c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:03.264672 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.264897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:03.265541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:03.265667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-a5089b80-d9ea-46a6-b9c1-5e0e05e3390d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.265849 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:03.266098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] PATH_INFO: `/v3/users/4b01aa2e35204ae9a3e30d98abf870f3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:03.267463 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b01aa2e35204ae9a3e30d98abf870f3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:03.267674 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1363/2686] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:03 2022] DELETE /identity/v3/projects/7ea33ee304ac4545898f04bc988da18c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:03.268938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33987b67-4bf4-4011-b74e-0b486be09da7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:03.269364 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33987b67-4bf4-4011-b74e-0b486be09da7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:03.269609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33987b67-4bf4-4011-b74e-0b486be09da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:03.269862 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:03.292283 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-33987b67-4bf4-4011-b74e-0b486be09da7 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:03.293506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-33987b67-4bf4-4011-b74e-0b486be09da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:03.302577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1364/2687] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:03.305840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33987b67-4bf4-4011-b74e-0b486be09da7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:03.305840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-33987b67-4bf4-4011-b74e-0b486be09da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:03.316384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33987b67-4bf4-4011-b74e-0b486be09da7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:03.321379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:03.322010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:03.322312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:03.322603 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] PATH_INFO: `/v3/users/3d6ae59e363c43e3ab77c24f51f48b03` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:03.324136 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d6ae59e363c43e3ab77c24f51f48b03)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:03.325676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:03.492841 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b01aa2e35204ae9a3e30d98abf870f3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.493501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b01aa2e35204ae9a3e30d98abf870f3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.505641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b01aa2e35204ae9a3e30d98abf870f3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.520237 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d6ae59e363c43e3ab77c24f51f48b03'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.532260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d6ae59e363c43e3ab77c24f51f48b03'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.532934 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d6ae59e363c43e3ab77c24f51f48b03'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.608610 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b01aa2e35204ae9a3e30d98abf870f3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.609280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b01aa2e35204ae9a3e30d98abf870f3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.622274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1324/2688] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:03 2022] DELETE /identity/v3/users/4b01aa2e35204ae9a3e30d98abf870f3 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:03.623063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:03.624373 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:03.665430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d6ae59e363c43e3ab77c24f51f48b03'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.666109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d6ae59e363c43e3ab77c24f51f48b03'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.679622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1365/2689] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:03 2022] DELETE /identity/v3/users/3d6ae59e363c43e3ab77c24f51f48b03 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:03.680594 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:03.681084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aaa88-cbbe-4a1b-a133-924d20f627cb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:03.682426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:03.735231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2635005-6b69-454c-a143-33a72bd9a662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:03.735845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2635005-6b69-454c-a143-33a72bd9a662 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:03.736242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2635005-6b69-454c-a143-33a72bd9a662 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:03.736393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2635005-6b69-454c-a143-33a72bd9a662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:03.738293 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b2635005-6b69-454c-a143-33a72bd9a662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:03.745073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:03.777969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b9310-246f-4137-ad3a-414ce83dac71 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:03.784557 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:03.785840 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:03.786126 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:03.786423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] PATH_INFO: `/v3/projects/775d4a7723de41c89a362a1460cc5f2f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:03.787785 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=775d4a7723de41c89a362a1460cc5f2f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:03.789981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:03.803869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2635005-6b69-454c-a143-33a72bd9a662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:03.805363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2635005-6b69-454c-a143-33a72bd9a662 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:03.815495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1325/2690] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:03.819977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2ede-f9f9-4030-8f85-7c367de95388 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:03.820263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2ede-f9f9-4030-8f85-7c367de95388 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:03.820536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2ede-f9f9-4030-8f85-7c367de95388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:03.842732 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b2db2ede-f9f9-4030-8f85-7c367de95388 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:03.843931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b2db2ede-f9f9-4030-8f85-7c367de95388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:03.852255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1326/2691] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:03.862784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db2ede-f9f9-4030-8f85-7c367de95388 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:03.865309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b2db2ede-f9f9-4030-8f85-7c367de95388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:03.880316 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db2ede-f9f9-4030-8f85-7c367de95388 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:03.882868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '775d4a7723de41c89a362a1460cc5f2f', 'request_id': 'req-7585e42a-29b4-41c7-97e5-0e9a236e4017'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.891532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '775d4a7723de41c89a362a1460cc5f2f', 'request_id': 'req-7585e42a-29b4-41c7-97e5-0e9a236e4017'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.961636 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c8441-0eff-4703-ba47-b1db7f73c3c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:03.962505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c8441-0eff-4703-ba47-b1db7f73c3c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:03.962982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c8441-0eff-4703-ba47-b1db7f73c3c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:03.963710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c8441-0eff-4703-ba47-b1db7f73c3c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:03.979717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '775d4a7723de41c89a362a1460cc5f2f', 'request_id': 'req-7585e42a-29b4-41c7-97e5-0e9a236e4017'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:03.981881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:03.998907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] The token cache is being invalidate because project 775d4a7723de41c89a362a1460cc5f2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:03.999321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.001215 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.003534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1366/2692] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:03 2022] DELETE /identity/v3/projects/775d4a7723de41c89a362a1460cc5f2f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:04.008419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:04.010402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:04.036578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c8441-0eff-4703-ba47-b1db7f73c3c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:04.038548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c8441-0eff-4703-ba47-b1db7f73c3c8 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:04.050750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1327/2693] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:04.102200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585e42a-29b4-41c7-97e5-0e9a236e4017 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:04.108142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:04.112316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:04.112316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:04.112316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] PATH_INFO: `/v3/users/687847b45aba44c8838ab953cdcabe69` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:04.112316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=687847b45aba44c8838ab953cdcabe69)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:04.113216 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:04.176036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c8441-0eff-4703-ba47-b1db7f73c3c8 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:04.178059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f41c8441-0eff-4703-ba47-b1db7f73c3c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:04.192557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c8441-0eff-4703-ba47-b1db7f73c3c8 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:04.199697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:04.201455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:04.201743 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:04.202019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] PATH_INFO: `/v3/projects/b0741ca2f5b14118b25959b859ee54c3` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:04.203722 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0741ca2f5b14118b25959b859ee54c3)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:04.206186 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:04.415594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:04.419739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '687847b45aba44c8838ab953cdcabe69'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.421616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0741ca2f5b14118b25959b859ee54c3', 'request_id': 'req-e28688e7-299b-4ca9-b70f-53a9106348e9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.429262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0741ca2f5b14118b25959b859ee54c3', 'request_id': 'req-e28688e7-299b-4ca9-b70f-53a9106348e9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.435229 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '687847b45aba44c8838ab953cdcabe69'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.436170 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '687847b45aba44c8838ab953cdcabe69'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.494194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0741ca2f5b14118b25959b859ee54c3', 'request_id': 'req-e28688e7-299b-4ca9-b70f-53a9106348e9'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.511470 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] The token cache is being invalidate because project b0741ca2f5b14118b25959b859ee54c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:04.511829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.513473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.515395 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1328/2694] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:04 2022] DELETE /identity/v3/projects/b0741ca2f5b14118b25959b859ee54c3 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:04.515960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:04.517285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:04.557223 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '687847b45aba44c8838ab953cdcabe69'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.558636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-5e49265c-506c-4a5c-ac87-4a1a64bb28f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '687847b45aba44c8838ab953cdcabe69'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.576185 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1367/2695] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:04 2022] DELETE /identity/v3/users/687847b45aba44c8838ab953cdcabe69 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:04.579040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986072c6-62db-4ea0-891b-053cb2d73af8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:04.579740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986072c6-62db-4ea0-891b-053cb2d73af8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:04.579740 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986072c6-62db-4ea0-891b-053cb2d73af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:04.589857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28688e7-299b-4ca9-b70f-53a9106348e9 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:04.603296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-986072c6-62db-4ea0-891b-053cb2d73af8 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:04.604911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-986072c6-62db-4ea0-891b-053cb2d73af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:04.657699 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:04.658615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:04.658943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:04.659392 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] PATH_INFO: `/v3/projects/cf249d61ef5549fe8c9ead7763436614` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:04.660634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf249d61ef5549fe8c9ead7763436614)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:04.663757 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:04.678129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1368/2696] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:04.685965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa56a4-e83c-4edf-81ca-11896fba428f None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:04.686307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa56a4-e83c-4edf-81ca-11896fba428f None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:04.686963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa56a4-e83c-4edf-81ca-11896fba428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:04.705673 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-a3fa56a4-e83c-4edf-81ca-11896fba428f None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:04.706888 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a3fa56a4-e83c-4edf-81ca-11896fba428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:04.715311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1369/2697] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:51:04.723138 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa56a4-e83c-4edf-81ca-11896fba428f None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:04.731296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa56a4-e83c-4edf-81ca-11896fba428f None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:04.736400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464ceca-9c17-4caa-9cc4-e7c823826c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:04.737003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464ceca-9c17-4caa-9cc4-e7c823826c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:04.737278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464ceca-9c17-4caa-9cc4-e7c823826c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:04.737548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464ceca-9c17-4caa-9cc4-e7c823826c4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:04.738420 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464ceca-9c17-4caa-9cc4-e7c823826c4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:51:04.738723 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464ceca-9c17-4caa-9cc4-e7c823826c4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:04.740368 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b464ceca-9c17-4caa-9cc4-e7c823826c4b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:04.752708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1370/2698] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 05:51:04.757016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464ceca-9c17-4caa-9cc4-e7c823826c4b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:04.758950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b464ceca-9c17-4caa-9cc4-e7c823826c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:04.778482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464ceca-9c17-4caa-9cc4-e7c823826c4b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:04.780949 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:04.781596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:04.781874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:04.782164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] PATH_INFO: `/v3/projects/fb5195738e9a49da8a1dcd11426e2d4c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:04.784382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb5195738e9a49da8a1dcd11426e2d4c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:04.788135 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:04.823977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf249d61ef5549fe8c9ead7763436614', 'request_id': 'req-41758df9-1800-4a1a-adfa-eca580d52d94'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.825897 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:04.830536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf249d61ef5549fe8c9ead7763436614', 'request_id': 'req-41758df9-1800-4a1a-adfa-eca580d52d94'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.918930 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf249d61ef5549fe8c9ead7763436614', 'request_id': 'req-41758df9-1800-4a1a-adfa-eca580d52d94'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.936059 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] The token cache is being invalidate because project cf249d61ef5549fe8c9ead7763436614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:04.936461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.937922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.944560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1329/2699] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:04 2022] DELETE /identity/v3/projects/cf249d61ef5549fe8c9ead7763436614 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:04.944560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:04.947835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb5195738e9a49da8a1dcd11426e2d4c', 'request_id': 'req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.948591 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:04.956902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb5195738e9a49da8a1dcd11426e2d4c', 'request_id': 'req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:04.959461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:05.018182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41758df9-1800-4a1a-adfa-eca580d52d94 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.019117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb5195738e9a49da8a1dcd11426e2d4c', 'request_id': 'req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:05.034035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] The token cache is being invalidate because project fb5195738e9a49da8a1dcd11426e2d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:05.034512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:05.035231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f2a8bab0-ee0e-4864-bc2a-74cc634170ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:05.037653 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1371/2700] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:04 2022] DELETE /identity/v3/projects/fb5195738e9a49da8a1dcd11426e2d4c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:05.043225 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6edc8-a530-4eb9-a20e-8150ede43a81 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.043512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6edc8-a530-4eb9-a20e-8150ede43a81 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.043784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6edc8-a530-4eb9-a20e-8150ede43a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.066166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-9ca6edc8-a530-4eb9-a20e-8150ede43a81 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:05.067641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-9ca6edc8-a530-4eb9-a20e-8150ede43a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.084383 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719314a-371d-4bf6-853a-d55bf5cc90d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.084919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719314a-371d-4bf6-853a-d55bf5cc90d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.085329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719314a-371d-4bf6-853a-d55bf5cc90d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.085616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719314a-371d-4bf6-853a-d55bf5cc90d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.086283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719314a-371d-4bf6-853a-d55bf5cc90d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.088187 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719314a-371d-4bf6-853a-d55bf5cc90d3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.133571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1372/2701] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:05.134880 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecff7f2-b70e-4798-b94f-ef94ac7d4dca None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.135166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecff7f2-b70e-4798-b94f-ef94ac7d4dca None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.135437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecff7f2-b70e-4798-b94f-ef94ac7d4dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.155964 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-aecff7f2-b70e-4798-b94f-ef94ac7d4dca None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:05.157192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aecff7f2-b70e-4798-b94f-ef94ac7d4dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.165576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1373/2702] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:05.166833 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bb139-fab4-4488-a453-89264178917d None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.167713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bb139-fab4-4488-a453-89264178917d None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.168955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bb139-fab4-4488-a453-89264178917d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.188648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e81bb139-fab4-4488-a453-89264178917d None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:05.189861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e81bb139-fab4-4488-a453-89264178917d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.202306 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1330/2703] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:04 2022] POST /identity/v3/projects => generated 373 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:05.202458 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1374/2704] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:05.203491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bb139-fab4-4488-a453-89264178917d None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.203928 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14742a-7f6b-4317-8487-c098b6bf8795 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.204221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14742a-7f6b-4317-8487-c098b6bf8795 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.204501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14742a-7f6b-4317-8487-c098b6bf8795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.206533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e81bb139-fab4-4488-a453-89264178917d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.219849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bb139-fab4-4488-a453-89264178917d None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.231563 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-0f14742a-7f6b-4317-8487-c098b6bf8795 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:05.232821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0f14742a-7f6b-4317-8487-c098b6bf8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.303893 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.304530 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.304927 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.305209 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] PATH_INFO: `/v3/projects/1392d5333d80426588ff5c5c189e2c38` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.306376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1392d5333d80426588ff5c5c189e2c38)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.308504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.313019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1331/2705] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:05.313781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14742a-7f6b-4317-8487-c098b6bf8795 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.315575 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-0f14742a-7f6b-4317-8487-c098b6bf8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.328927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14742a-7f6b-4317-8487-c098b6bf8795 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.335872 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3431ec-8eb7-4e3c-96a7-2f242d349227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.336762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3431ec-8eb7-4e3c-96a7-2f242d349227 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.337042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3431ec-8eb7-4e3c-96a7-2f242d349227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.337594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3431ec-8eb7-4e3c-96a7-2f242d349227 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.337944 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3431ec-8eb7-4e3c-96a7-2f242d349227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.339923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3431ec-8eb7-4e3c-96a7-2f242d349227 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.415959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1392d5333d80426588ff5c5c189e2c38', 'request_id': 'req-65f20ff5-00e5-40ac-8c01-c9f0995863b7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:05.422537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1392d5333d80426588ff5c5c189e2c38', 'request_id': 'req-65f20ff5-00e5-40ac-8c01-c9f0995863b7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:05.430241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1332/2706] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:05.430241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3431ec-8eb7-4e3c-96a7-2f242d349227 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.438471 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3431ec-8eb7-4e3c-96a7-2f242d349227 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.495487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c11490-7c3a-4e15-8e32-4c4e281e15cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.496109 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c11490-7c3a-4e15-8e32-4c4e281e15cd admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.496394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c11490-7c3a-4e15-8e32-4c4e281e15cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.496670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c11490-7c3a-4e15-8e32-4c4e281e15cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.497479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c11490-7c3a-4e15-8e32-4c4e281e15cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:51:05.497782 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c11490-7c3a-4e15-8e32-4c4e281e15cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.499401 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c11490-7c3a-4e15-8e32-4c4e281e15cd admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.512002 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1333/2707] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:51:05.513326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c11490-7c3a-4e15-8e32-4c4e281e15cd admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.523047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c11490-7c3a-4e15-8e32-4c4e281e15cd admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.527822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba85c08-89f2-4945-a94c-5f6b6b200f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.528404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba85c08-89f2-4945-a94c-5f6b6b200f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.528688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba85c08-89f2-4945-a94c-5f6b6b200f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.528962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba85c08-89f2-4945-a94c-5f6b6b200f9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.529599 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba85c08-89f2-4945-a94c-5f6b6b200f9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:51:05.529890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba85c08-89f2-4945-a94c-5f6b6b200f9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.531499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba85c08-89f2-4945-a94c-5f6b6b200f9a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.534004 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1392d5333d80426588ff5c5c189e2c38', 'request_id': 'req-65f20ff5-00e5-40ac-8c01-c9f0995863b7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:05.546051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1334/2708] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:51:05.546962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] The token cache is being invalidate because project 1392d5333d80426588ff5c5c189e2c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:05.547085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba85c08-89f2-4945-a94c-5f6b6b200f9a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.547260 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:05.547938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:05.553207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1375/2709] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:05 2022] DELETE /identity/v3/projects/1392d5333d80426588ff5c5c189e2c38 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:05.553207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.553207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.558959 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba85c08-89f2-4945-a94c-5f6b6b200f9a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.563786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0f6c2-1591-40e8-a23f-0d502984b5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.564390 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0f6c2-1591-40e8-a23f-0d502984b5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.564669 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0f6c2-1591-40e8-a23f-0d502984b5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.564941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0f6c2-1591-40e8-a23f-0d502984b5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.565317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f20ff5-00e5-40ac-8c01-c9f0995863b7 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.565761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0f6c2-1591-40e8-a23f-0d502984b5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.567447 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0f6c2-1591-40e8-a23f-0d502984b5e1 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.571523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474ad61-64a2-428a-aa25-9d6a8cdbe75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.572542 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474ad61-64a2-428a-aa25-9d6a8cdbe75b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.572542 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474ad61-64a2-428a-aa25-9d6a8cdbe75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.572798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474ad61-64a2-428a-aa25-9d6a8cdbe75b admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.573421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474ad61-64a2-428a-aa25-9d6a8cdbe75b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.575682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474ad61-64a2-428a-aa25-9d6a8cdbe75b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.579010 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1335/2710] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:05.580117 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0f6c2-1591-40e8-a23f-0d502984b5e1 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.581805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-38c0f6c2-1591-40e8-a23f-0d502984b5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.599422 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:05.599615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0f6c2-1591-40e8-a23f-0d502984b5e1 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.601356 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43f0dc-af7e-40a4-96a0-5337edff9026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.601953 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43f0dc-af7e-40a4-96a0-5337edff9026 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.602268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43f0dc-af7e-40a4-96a0-5337edff9026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.602543 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43f0dc-af7e-40a4-96a0-5337edff9026 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.603178 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43f0dc-af7e-40a4-96a0-5337edff9026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.605056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43f0dc-af7e-40a4-96a0-5337edff9026 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.659794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:05.667306 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1376/2711] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:05.668742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe377a6-81fc-4e0d-942c-91f3a109a178 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.671371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe377a6-81fc-4e0d-942c-91f3a109a178 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.671371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe377a6-81fc-4e0d-942c-91f3a109a178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.689786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7fe377a6-81fc-4e0d-942c-91f3a109a178 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:05.691027 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7fe377a6-81fc-4e0d-942c-91f3a109a178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.707528 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1336/2712] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:05.708599 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43f0dc-af7e-40a4-96a0-5337edff9026 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.710397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ee43f0dc-af7e-40a4-96a0-5337edff9026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.764739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1377/2713] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:05.771980 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe377a6-81fc-4e0d-942c-91f3a109a178 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.778609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43f0dc-af7e-40a4-96a0-5337edff9026 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.783382 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994888eb-6224-49c2-ae57-260aa5301898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.784004 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994888eb-6224-49c2-ae57-260aa5301898 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.784572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994888eb-6224-49c2-ae57-260aa5301898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.784572 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994888eb-6224-49c2-ae57-260aa5301898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.785108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe377a6-81fc-4e0d-942c-91f3a109a178 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.785217 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994888eb-6224-49c2-ae57-260aa5301898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.786906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994888eb-6224-49c2-ae57-260aa5301898 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.790846 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faedc7c-3278-4a1c-a8d3-cb3016ad814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.791465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faedc7c-3278-4a1c-a8d3-cb3016ad814f admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.792426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faedc7c-3278-4a1c-a8d3-cb3016ad814f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.792732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faedc7c-3278-4a1c-a8d3-cb3016ad814f admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.793627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faedc7c-3278-4a1c-a8d3-cb3016ad814f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.795702 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faedc7c-3278-4a1c-a8d3-cb3016ad814f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.797043 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1337/2714] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:05.801136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994888eb-6224-49c2-ae57-260aa5301898 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.811623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994888eb-6224-49c2-ae57-260aa5301898 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.817224 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76daa0-015e-4d8a-8904-7c9eb09bac15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.817822 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76daa0-015e-4d8a-8904-7c9eb09bac15 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.818708 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76daa0-015e-4d8a-8904-7c9eb09bac15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.819423 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76daa0-015e-4d8a-8904-7c9eb09bac15 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.820400 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76daa0-015e-4d8a-8904-7c9eb09bac15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.822587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76daa0-015e-4d8a-8904-7c9eb09bac15 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.918800 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1378/2715] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:05.920239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead20ce-92cf-4a8c-91f4-3c0299e250c6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.920512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead20ce-92cf-4a8c-91f4-3c0299e250c6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.920807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead20ce-92cf-4a8c-91f4-3c0299e250c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.943386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-8ead20ce-92cf-4a8c-91f4-3c0299e250c6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:05.944433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1338/2716] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:05.944783 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-8ead20ce-92cf-4a8c-91f4-3c0299e250c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:05.945357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76daa0-015e-4d8a-8904-7c9eb09bac15 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.953607 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1379/2717] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:05.954409 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead20ce-92cf-4a8c-91f4-3c0299e250c6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:05.958817 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76daa0-015e-4d8a-8904-7c9eb09bac15 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.963854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2d403-3efd-479d-a174-f389a0eaf116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.964599 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2d403-3efd-479d-a174-f389a0eaf116 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.964940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2d403-3efd-479d-a174-f389a0eaf116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.969303 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2d403-3efd-479d-a174-f389a0eaf116 admin admin] PATH_INFO: `/v3/projects/a1efce1ebdf1425396c2babae74fe6ce/users/e04f49a794c5487ba372cf1baaf67402/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.969431 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead20ce-92cf-4a8c-91f4-3c0299e250c6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:05.973898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd7576-1967-4f26-8b08-e1faa69ae43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:05.975212 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd7576-1967-4f26-8b08-e1faa69ae43c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:05.976239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd7576-1967-4f26-8b08-e1faa69ae43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:05.976523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd7576-1967-4f26-8b08-e1faa69ae43c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:05.977570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd7576-1967-4f26-8b08-e1faa69ae43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.979032 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd7576-1967-4f26-8b08-e1faa69ae43c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.982913 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2d403-3efd-479d-a174-f389a0eaf116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1efce1ebdf1425396c2babae74fe6ce, user_id=e04f49a794c5487ba372cf1baaf67402, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:05.985925 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2d403-3efd-479d-a174-f389a0eaf116 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:05.994454 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1380/2718] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:05.995793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd7576-1967-4f26-8b08-e1faa69ae43c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.005379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd7576-1967-4f26-8b08-e1faa69ae43c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.010616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0c662-ef78-4d6f-9d00-fffac1dc7821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.011233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0c662-ef78-4d6f-9d00-fffac1dc7821 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.011638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0c662-ef78-4d6f-9d00-fffac1dc7821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.011916 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0c662-ef78-4d6f-9d00-fffac1dc7821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.012624 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0c662-ef78-4d6f-9d00-fffac1dc7821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.014281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0c662-ef78-4d6f-9d00-fffac1dc7821 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.024542 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1381/2719] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:06.025362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0c662-ef78-4d6f-9d00-fffac1dc7821 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.035045 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0c662-ef78-4d6f-9d00-fffac1dc7821 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.040533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0d094-092d-404a-8f2c-fbeb59db587c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.041437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0d094-092d-404a-8f2c-fbeb59db587c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.041437 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0d094-092d-404a-8f2c-fbeb59db587c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.041742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0d094-092d-404a-8f2c-fbeb59db587c admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.042385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0d094-092d-404a-8f2c-fbeb59db587c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:51:06.042690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0d094-092d-404a-8f2c-fbeb59db587c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.044281 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0d094-092d-404a-8f2c-fbeb59db587c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.049553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1339/2720] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:05 2022] PUT /identity/v3/projects/a1efce1ebdf1425396c2babae74fe6ce/users/e04f49a794c5487ba372cf1baaf67402/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:06.053967 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2d403-3efd-479d-a174-f389a0eaf116 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.065399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1382/2721] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:51:06.065399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0d094-092d-404a-8f2c-fbeb59db587c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.073031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0d094-092d-404a-8f2c-fbeb59db587c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.123251 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2d403-3efd-479d-a174-f389a0eaf116 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.128201 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9154b-f67d-4e41-9a7c-6cfd05d98764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.128813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9154b-f67d-4e41-9a7c-6cfd05d98764 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.129094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9154b-f67d-4e41-9a7c-6cfd05d98764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.129371 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9154b-f67d-4e41-9a7c-6cfd05d98764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.130028 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9154b-f67d-4e41-9a7c-6cfd05d98764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.134795 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9154b-f67d-4e41-9a7c-6cfd05d98764 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.136345 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45372fe-a47a-43fb-a336-0ea34a86fa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.137147 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45372fe-a47a-43fb-a336-0ea34a86fa44 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.137429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45372fe-a47a-43fb-a336-0ea34a86fa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.137743 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45372fe-a47a-43fb-a336-0ea34a86fa44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.138462 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45372fe-a47a-43fb-a336-0ea34a86fa44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.140174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45372fe-a47a-43fb-a336-0ea34a86fa44 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.145342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1340/2722] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:06.145342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90811fe-18fd-43ed-ba6d-6f8f1e8a7d29 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.145342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90811fe-18fd-43ed-ba6d-6f8f1e8a7d29 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.145342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90811fe-18fd-43ed-ba6d-6f8f1e8a7d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.155503 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1383/2723] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:06.156267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45372fe-a47a-43fb-a336-0ea34a86fa44 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.167218 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45372fe-a47a-43fb-a336-0ea34a86fa44 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.167867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-a90811fe-18fd-43ed-ba6d-6f8f1e8a7d29 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:06.169451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a90811fe-18fd-43ed-ba6d-6f8f1e8a7d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:06.171960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f9568-f039-4cd4-a220-a07fc70b74ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.172604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f9568-f039-4cd4-a220-a07fc70b74ba admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.172960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f9568-f039-4cd4-a220-a07fc70b74ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.173162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f9568-f039-4cd4-a220-a07fc70b74ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.173796 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f9568-f039-4cd4-a220-a07fc70b74ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.175032 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:06.175816 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f9568-f039-4cd4-a220-a07fc70b74ba admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.183056 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1341/2724] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:06.187103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90811fe-18fd-43ed-ba6d-6f8f1e8a7d29 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.197566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90811fe-18fd-43ed-ba6d-6f8f1e8a7d29 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.203204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c90e0-1de9-4522-8bde-eecb37771a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.203674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c90e0-1de9-4522-8bde-eecb37771a90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.206522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c90e0-1de9-4522-8bde-eecb37771a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.206522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c90e0-1de9-4522-8bde-eecb37771a90 admin admin] PATH_INFO: `/v3/projects/55a23449c88b45f68be04bcdcd1e7a98/users/4ebc957cf725466f92ef2238a0eadf20/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.220220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c90e0-1de9-4522-8bde-eecb37771a90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a23449c88b45f68be04bcdcd1e7a98, user_id=4ebc957cf725466f92ef2238a0eadf20, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.223150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c90e0-1de9-4522-8bde-eecb37771a90 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.253009 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1384/2725] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:06 2022] POST /identity/v3/projects => generated 401 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:06.253009 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f9568-f039-4cd4-a220-a07fc70b74ba admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.318479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f9568-f039-4cd4-a220-a07fc70b74ba admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.323370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1342/2726] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:06 2022] PUT /identity/v3/projects/55a23449c88b45f68be04bcdcd1e7a98/users/4ebc957cf725466f92ef2238a0eadf20/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:06.333450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c90e0-1de9-4522-8bde-eecb37771a90 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.333450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ac2c90e0-1de9-4522-8bde-eecb37771a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:06.389402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c437a2c-fb41-4a09-bfad-8eeae7e34c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.390072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c437a2c-fb41-4a09-bfad-8eeae7e34c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.390382 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c437a2c-fb41-4a09-bfad-8eeae7e34c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.390660 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c437a2c-fb41-4a09-bfad-8eeae7e34c73 admin admin] PATH_INFO: `/v3/projects/bb52515b3bb846b39ca83b99744c5692/users/33e51bb597844cea82394afaad18eaaa/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.393838 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c90e0-1de9-4522-8bde-eecb37771a90 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.406239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dcb60-a710-47a9-bd08-d27ab72e428d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.406239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dcb60-a710-47a9-bd08-d27ab72e428d service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.406239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dcb60-a710-47a9-bd08-d27ab72e428d service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.406239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dcb60-a710-47a9-bd08-d27ab72e428d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.406902 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c437a2c-fb41-4a09-bfad-8eeae7e34c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb52515b3bb846b39ca83b99744c5692, user_id=33e51bb597844cea82394afaad18eaaa, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.409348 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908dcb60-a710-47a9-bd08-d27ab72e428d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.409728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c437a2c-fb41-4a09-bfad-8eeae7e34c73 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.410789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908dcb60-a710-47a9-bd08-d27ab72e428d service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.421501 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1343/2727] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:06.422346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dcb60-a710-47a9-bd08-d27ab72e428d service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.425402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:06.433640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dcb60-a710-47a9-bd08-d27ab72e428d service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.439553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432ca19-2d96-40fc-9296-700afdbd8892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.440155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432ca19-2d96-40fc-9296-700afdbd8892 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.440432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432ca19-2d96-40fc-9296-700afdbd8892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.440705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432ca19-2d96-40fc-9296-700afdbd8892 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.441322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432ca19-2d96-40fc-9296-700afdbd8892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.443530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432ca19-2d96-40fc-9296-700afdbd8892 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.483031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1385/2728] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:06 2022] PUT /identity/v3/projects/bb52515b3bb846b39ca83b99744c5692/users/33e51bb597844cea82394afaad18eaaa/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:06.486755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406de1f-0b69-4128-8eae-42f3c8992ae7 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.486755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406de1f-0b69-4128-8eae-42f3c8992ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.486755 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406de1f-0b69-4128-8eae-42f3c8992ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.519433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-1406de1f-0b69-4128-8eae-42f3c8992ae7 None None] MFA Rules not processed for user `4ebc957cf725466f92ef2238a0eadf20`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:06.521776 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1406de1f-0b69-4128-8eae-42f3c8992ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:06.569113 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1344/2729] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:06 2022] POST /identity/v3/users => generated 301 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:06.570236 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4f629-6750-4ced-b0d3-0216734fa48e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.570522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4f629-6750-4ced-b0d3-0216734fa48e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.570802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4f629-6750-4ced-b0d3-0216734fa48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.626245 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-aed4f629-6750-4ced-b0d3-0216734fa48e None None] MFA Rules not processed for user `33e51bb597844cea82394afaad18eaaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:06.627396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1386/2730] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:06.627845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aed4f629-6750-4ced-b0d3-0216734fa48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:06.628895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406de1f-0b69-4128-8eae-42f3c8992ae7 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.733969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406de1f-0b69-4128-8eae-42f3c8992ae7 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.738994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1345/2731] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:06.740726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4f629-6750-4ced-b0d3-0216734fa48e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.744424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-aed4f629-6750-4ced-b0d3-0216734fa48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:06.744823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c96104-84ea-4961-8bbc-ff6a33331663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.745496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c96104-84ea-4961-8bbc-ff6a33331663 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.745784 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c96104-84ea-4961-8bbc-ff6a33331663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.746061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c96104-84ea-4961-8bbc-ff6a33331663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.748385 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c96104-84ea-4961-8bbc-ff6a33331663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.753342 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c96104-84ea-4961-8bbc-ff6a33331663 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.753807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:06.765169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1387/2732] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:06.766039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c96104-84ea-4961-8bbc-ff6a33331663 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.778691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c96104-84ea-4961-8bbc-ff6a33331663 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.785035 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b66531-01c2-4b34-aa99-be571bcc024c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.785713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b66531-01c2-4b34-aa99-be571bcc024c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.786000 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b66531-01c2-4b34-aa99-be571bcc024c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.786309 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b66531-01c2-4b34-aa99-be571bcc024c admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.787026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b66531-01c2-4b34-aa99-be571bcc024c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.789014 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b66531-01c2-4b34-aa99-be571bcc024c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.826452 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4f629-6750-4ced-b0d3-0216734fa48e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.830950 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df25551-4493-4b09-8012-ebe8527e9bec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34eb8fcef0543c282b3887dfeb74c15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.831569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df25551-4493-4b09-8012-ebe8527e9bec service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.831995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df25551-4493-4b09-8012-ebe8527e9bec service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.832635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df25551-4493-4b09-8012-ebe8527e9bec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.842931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df25551-4493-4b09-8012-ebe8527e9bec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.844709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df25551-4493-4b09-8012-ebe8527e9bec service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.856940 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1346/2733] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:06.859784 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df25551-4493-4b09-8012-ebe8527e9bec service glance] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.874051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df25551-4493-4b09-8012-ebe8527e9bec service glance] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:06.881357 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34777de9-a03a-46cd-874a-5595dc4f123d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:06.883987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777de9-a03a-46cd-874a-5595dc4f123d admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:06.884706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777de9-a03a-46cd-874a-5595dc4f123d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:06.886199 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777de9-a03a-46cd-874a-5595dc4f123d admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:06.887352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34777de9-a03a-46cd-874a-5595dc4f123d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:06.889368 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34777de9-a03a-46cd-874a-5595dc4f123d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:06.894799 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1388/2734] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:06 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:06.897843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b66531-01c2-4b34-aa99-be571bcc024c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.925431 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1347/2735] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:06.930861 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34777de9-a03a-46cd-874a-5595dc4f123d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:06.958258 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34777de9-a03a-46cd-874a-5595dc4f123d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.007256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b66531-01c2-4b34-aa99-be571bcc024c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.011332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:07.024684 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68089aca-6569-4d58-bace-55098a2b3e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.026334 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68089aca-6569-4d58-bace-55098a2b3e25 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.027263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68089aca-6569-4d58-bace-55098a2b3e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.027810 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68089aca-6569-4d58-bace-55098a2b3e25 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.028822 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68089aca-6569-4d58-bace-55098a2b3e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:07.040793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68089aca-6569-4d58-bace-55098a2b3e25 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:07.063964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787bb98-f4a8-422d-a62b-5d4714b0db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.064645 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787bb98-f4a8-422d-a62b-5d4714b0db14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.064920 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787bb98-f4a8-422d-a62b-5d4714b0db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.065190 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787bb98-f4a8-422d-a62b-5d4714b0db14 admin admin] PATH_INFO: `/v3/projects/9b6f212db63f480fa2a4c13e47f6a5bb/users/c42af0abf5054019b8924ae08b892b17/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.085932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787bb98-f4a8-422d-a62b-5d4714b0db14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6f212db63f480fa2a4c13e47f6a5bb, user_id=c42af0abf5054019b8924ae08b892b17, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:07.088844 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787bb98-f4a8-422d-a62b-5d4714b0db14 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:07.211058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1348/2736] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:06 2022] PUT /identity/v3/projects/9b6f212db63f480fa2a4c13e47f6a5bb/users/c42af0abf5054019b8924ae08b892b17/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:07.224106 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1389/2737] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:06 2022] POST /identity/v3/users => generated 297 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:07.224892 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d724c-ada1-4ec0-ab9a-4b0026dfbdf8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.224892 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d724c-ada1-4ec0-ab9a-4b0026dfbdf8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.224892 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d724c-ada1-4ec0-ab9a-4b0026dfbdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.227495 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68089aca-6569-4d58-bace-55098a2b3e25 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:07.242040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68089aca-6569-4d58-bace-55098a2b3e25 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.308938 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-4a2d724c-ada1-4ec0-ab9a-4b0026dfbdf8 None None] MFA Rules not processed for user `c42af0abf5054019b8924ae08b892b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:07.310196 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4a2d724c-ada1-4ec0-ab9a-4b0026dfbdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:07.327537 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657ac81-e08f-45fc-829b-15dc7b3b04ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.328200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657ac81-e08f-45fc-829b-15dc7b3b04ec admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.329157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657ac81-e08f-45fc-829b-15dc7b3b04ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.329488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657ac81-e08f-45fc-829b-15dc7b3b04ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.330709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657ac81-e08f-45fc-829b-15dc7b3b04ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:07.332933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657ac81-e08f-45fc-829b-15dc7b3b04ec admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:07.348907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1390/2738] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:51:07.350548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657ac81-e08f-45fc-829b-15dc7b3b04ec admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:07.361328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657ac81-e08f-45fc-829b-15dc7b3b04ec admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.366977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46e475-3773-41eb-9b10-6063689dce9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.367948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46e475-3773-41eb-9b10-6063689dce9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.368231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46e475-3773-41eb-9b10-6063689dce9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.368509 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46e475-3773-41eb-9b10-6063689dce9b admin admin] PATH_INFO: `/v3/projects/d498116cdb0d485291403b1d8e33f9fa/users/7253ebb304ed4c60a3da0983285875c7/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.389548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46e475-3773-41eb-9b10-6063689dce9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d498116cdb0d485291403b1d8e33f9fa, user_id=7253ebb304ed4c60a3da0983285875c7, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:07.393745 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46e475-3773-41eb-9b10-6063689dce9b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:07.422500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1349/2739] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:07 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:07.427010 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d724c-ada1-4ec0-ab9a-4b0026dfbdf8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:07.438989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d724c-ada1-4ec0-ab9a-4b0026dfbdf8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.447689 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b790-80b1-4ea4-8da3-58475f0810d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.448308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b790-80b1-4ea4-8da3-58475f0810d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.448589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b790-80b1-4ea4-8da3-58475f0810d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.448895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b790-80b1-4ea4-8da3-58475f0810d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.449550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917b790-80b1-4ea4-8da3-58475f0810d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:07.451536 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917b790-80b1-4ea4-8da3-58475f0810d7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:07.457807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:07.465015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:07.465015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1391/2740] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:07 2022] PUT /identity/v3/projects/d498116cdb0d485291403b1d8e33f9fa/users/7253ebb304ed4c60a3da0983285875c7/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:07.472277 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46e475-3773-41eb-9b10-6063689dce9b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:07.550829 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46e475-3773-41eb-9b10-6063689dce9b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.555350 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1350/2741] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:07 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:07.559207 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b790-80b1-4ea4-8da3-58475f0810d7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:07.560142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d6857-78c8-4ecd-9981-f718739412ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.560766 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6857-78c8-4ecd-9981-f718739412ef admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.561097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6857-78c8-4ecd-9981-f718739412ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.561379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6857-78c8-4ecd-9981-f718739412ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.562024 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d6857-78c8-4ecd-9981-f718739412ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:07.563668 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d6857-78c8-4ecd-9981-f718739412ef admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:07.590890 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1392/2742] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:51:07.595146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d6857-78c8-4ecd-9981-f718739412ef admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:07.608011 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d6857-78c8-4ecd-9981-f718739412ef admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.673168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b790-80b1-4ea4-8da3-58475f0810d7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.681865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc903af-99cc-43ce-9f5f-52e818733c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.682517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc903af-99cc-43ce-9f5f-52e818733c29 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.682806 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc903af-99cc-43ce-9f5f-52e818733c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.683085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc903af-99cc-43ce-9f5f-52e818733c29 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.687499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc903af-99cc-43ce-9f5f-52e818733c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:07.687648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaec9d-efbb-433f-bedb-9b9d0dfb6bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.687648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaec9d-efbb-433f-bedb-9b9d0dfb6bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.687648 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaec9d-efbb-433f-bedb-9b9d0dfb6bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.688283 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc903af-99cc-43ce-9f5f-52e818733c29 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:07.688391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaec9d-efbb-433f-bedb-9b9d0dfb6bbe admin admin] PATH_INFO: `/v3/projects/d498116cdb0d485291403b1d8e33f9fa/users/7253ebb304ed4c60a3da0983285875c7/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.688634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaec9d-efbb-433f-bedb-9b9d0dfb6bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d498116cdb0d485291403b1d8e33f9fa, user_id=7253ebb304ed4c60a3da0983285875c7, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:07.691788 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaec9d-efbb-433f-bedb-9b9d0dfb6bbe admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:07.708019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1393/2743] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:07 2022] PUT /identity/v3/projects/d498116cdb0d485291403b1d8e33f9fa/users/7253ebb304ed4c60a3da0983285875c7/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:07.720410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e53e4-90a0-4485-ad86-f56771155c44 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.720707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e53e4-90a0-4485-ad86-f56771155c44 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.720966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e53e4-90a0-4485-ad86-f56771155c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.754336 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4b2e53e4-90a0-4485-ad86-f56771155c44 None None] MFA Rules not processed for user `7253ebb304ed4c60a3da0983285875c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:07.756505 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4b2e53e4-90a0-4485-ad86-f56771155c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:07.823182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1351/2744] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:07 2022] POST /identity/v3/users => generated 306 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:07.824958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc903af-99cc-43ce-9f5f-52e818733c29 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:07.834375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1394/2745] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:51:07.853365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e53e4-90a0-4485-ad86-f56771155c44 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:07.854889 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4b2e53e4-90a0-4485-ad86-f56771155c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:07.862028 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc903af-99cc-43ce-9f5f-52e818733c29 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.954729 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7b36d-488a-45d6-9c4c-517ad9a004f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.955611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7b36d-488a-45d6-9c4c-517ad9a004f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.955738 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7b36d-488a-45d6-9c4c-517ad9a004f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.956172 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7b36d-488a-45d6-9c4c-517ad9a004f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.956793 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7b36d-488a-45d6-9c4c-517ad9a004f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:07.958495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7b36d-488a-45d6-9c4c-517ad9a004f0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:07.968711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e53e4-90a0-4485-ad86-f56771155c44 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.973002 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccc5ce-121c-43ae-a70f-a48a8d1504e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.973182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1352/2746] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:07.976698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccc5ce-121c-43ae-a70f-a48a8d1504e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.977469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccc5ce-121c-43ae-a70f-a48a8d1504e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.977595 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7b36d-488a-45d6-9c4c-517ad9a004f0 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:07.977714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccc5ce-121c-43ae-a70f-a48a8d1504e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:07.987634 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7b36d-488a-45d6-9c4c-517ad9a004f0 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:07.988232 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccc5ce-121c-43ae-a70f-a48a8d1504e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:07.989814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccc5ce-121c-43ae-a70f-a48a8d1504e6 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:07.993717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8664de-96c4-46cc-8c14-361f5d501449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:07.994361 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8664de-96c4-46cc-8c14-361f5d501449 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:07.994714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8664de-96c4-46cc-8c14-361f5d501449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:07.995193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8664de-96c4-46cc-8c14-361f5d501449 admin admin] PATH_INFO: `/v3/projects/a5ae453fd20047daa63436fa1450f766/users/53d8737b398441c4bcffb9703eb43e5b/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:08.011708 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1395/2747] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:51:08.014103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8664de-96c4-46cc-8c14-361f5d501449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ae453fd20047daa63436fa1450f766, user_id=53d8737b398441c4bcffb9703eb43e5b, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:08.016962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8664de-96c4-46cc-8c14-361f5d501449 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:08.139583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1353/2748] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:07 2022] PUT /identity/v3/projects/a5ae453fd20047daa63436fa1450f766/users/53d8737b398441c4bcffb9703eb43e5b/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:08.149253 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccc5ce-121c-43ae-a70f-a48a8d1504e6 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:08.154792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccc5ce-121c-43ae-a70f-a48a8d1504e6 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:08.225492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca96f8d-b6e6-4e8b-9fdd-f58a81daaa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:08.226197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca96f8d-b6e6-4e8b-9fdd-f58a81daaa09 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:08.226468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca96f8d-b6e6-4e8b-9fdd-f58a81daaa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:08.226762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca96f8d-b6e6-4e8b-9fdd-f58a81daaa09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:08.227410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca96f8d-b6e6-4e8b-9fdd-f58a81daaa09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:08.229010 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca96f8d-b6e6-4e8b-9fdd-f58a81daaa09 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:08.239753 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1396/2749] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:08.244755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8664de-96c4-46cc-8c14-361f5d501449 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:08.255313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8664de-96c4-46cc-8c14-361f5d501449 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:08.267205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda170f-9b6a-499b-8b1b-89e96b67bf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:08.267205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda170f-9b6a-499b-8b1b-89e96b67bf1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:08.267205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda170f-9b6a-499b-8b1b-89e96b67bf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:08.267205 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda170f-9b6a-499b-8b1b-89e96b67bf1e admin admin] PATH_INFO: `/v3/projects/a5ae453fd20047daa63436fa1450f766/users/53d8737b398441c4bcffb9703eb43e5b/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:08.270094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda170f-9b6a-499b-8b1b-89e96b67bf1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ae453fd20047daa63436fa1450f766, user_id=53d8737b398441c4bcffb9703eb43e5b, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:08.273076 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda170f-9b6a-499b-8b1b-89e96b67bf1e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:08.293064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1354/2750] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:08 2022] PUT /identity/v3/projects/a5ae453fd20047daa63436fa1450f766/users/53d8737b398441c4bcffb9703eb43e5b/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:08.297722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca96f8d-b6e6-4e8b-9fdd-f58a81daaa09 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:08.308568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effec1-6340-4777-8f77-a3512b29f9c7 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:08.309000 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effec1-6340-4777-8f77-a3512b29f9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:08.310098 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effec1-6340-4777-8f77-a3512b29f9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:08.311898 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:08.352663 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-13effec1-6340-4777-8f77-a3512b29f9c7 None None] MFA Rules not processed for user `53d8737b398441c4bcffb9703eb43e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:08.353900 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-13effec1-6340-4777-8f77-a3512b29f9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:08.390683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:08.407506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca96f8d-b6e6-4e8b-9fdd-f58a81daaa09 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:08.438647 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1355/2751] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:08 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:08.442961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c0b85-3742-42fd-be89-19681101b8f4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:08.442961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c0b85-3742-42fd-be89-19681101b8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:08.442961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c0b85-3742-42fd-be89-19681101b8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:08.459109 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74299528-5fc9-43c5-b74f-99d383f60453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:08.459790 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74299528-5fc9-43c5-b74f-99d383f60453 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:08.460102 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74299528-5fc9-43c5-b74f-99d383f60453 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:08.460329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74299528-5fc9-43c5-b74f-99d383f60453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:08.470363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-387c0b85-3742-42fd-be89-19681101b8f4 None None] MFA Rules not processed for user `e04f49a794c5487ba372cf1baaf67402`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:08.471719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-387c0b85-3742-42fd-be89-19681101b8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:08.532173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74299528-5fc9-43c5-b74f-99d383f60453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:08.533696 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74299528-5fc9-43c5-b74f-99d383f60453 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:08.544769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1397/2752] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:51:08.545807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74299528-5fc9-43c5-b74f-99d383f60453 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:08.547415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-74299528-5fc9-43c5-b74f-99d383f60453 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:08.554985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1356/2753] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:08.562238 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74299528-5fc9-43c5-b74f-99d383f60453 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:08.566362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118a822-bfda-4597-b39d-ae84a357943a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:08.567433 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118a822-bfda-4597-b39d-ae84a357943a service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:08.567780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118a822-bfda-4597-b39d-ae84a357943a service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:08.568094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118a822-bfda-4597-b39d-ae84a357943a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:08.583141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118a822-bfda-4597-b39d-ae84a357943a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:08.585304 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118a822-bfda-4597-b39d-ae84a357943a service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:08.594912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1398/2754] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:08 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:08.691839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c0b85-3742-42fd-be89-19681101b8f4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:08.701984 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c0b85-3742-42fd-be89-19681101b8f4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:08.705616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d029ad-0e1a-436d-a64e-8938cf5c2210 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:08.706254 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d029ad-0e1a-436d-a64e-8938cf5c2210 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:08.706535 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d029ad-0e1a-436d-a64e-8938cf5c2210 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:08.706810 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d029ad-0e1a-436d-a64e-8938cf5c2210 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:08.716560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d029ad-0e1a-436d-a64e-8938cf5c2210 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:08.718031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d029ad-0e1a-436d-a64e-8938cf5c2210 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:08.728048 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1357/2755] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:08.792177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:09.110421 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118a822-bfda-4597-b39d-ae84a357943a service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:09.119628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118a822-bfda-4597-b39d-ae84a357943a service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:09.123438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bf99b-efd2-46ac-b088-17fa763d830b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:09.124031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bf99b-efd2-46ac-b088-17fa763d830b service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:09.124303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bf99b-efd2-46ac-b088-17fa763d830b service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:09.124570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bf99b-efd2-46ac-b088-17fa763d830b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:09.184054 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bf99b-efd2-46ac-b088-17fa763d830b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:09.190901 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bf99b-efd2-46ac-b088-17fa763d830b service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:09.195329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1399/2756] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:09 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:10.501946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b31b2-e400-469d-b64a-5238d9666bc0 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:10.502557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b31b2-e400-469d-b64a-5238d9666bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:10.502661 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b31b2-e400-469d-b64a-5238d9666bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:10.522718 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-cd0b31b2-e400-469d-b64a-5238d9666bc0 None None] MFA Rules not processed for user `e04f49a794c5487ba372cf1baaf67402`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:10.523911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-cd0b31b2-e400-469d-b64a-5238d9666bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:10.530957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1358/2757] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:12.721978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bf99b-efd2-46ac-b088-17fa763d830b service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:12.723491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7a4bf99b-efd2-46ac-b088-17fa763d830b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:12.736814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:12.827101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bf99b-efd2-46ac-b088-17fa763d830b service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:12.832478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e9e5b-7b87-48f6-90b4-d2c786da6239 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:12.833110 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e9e5b-7b87-48f6-90b4-d2c786da6239 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:12.833441 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e9e5b-7b87-48f6-90b4-d2c786da6239 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:12.835933 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e9e5b-7b87-48f6-90b4-d2c786da6239 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:12.836150 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-125e9e5b-7b87-48f6-90b4-d2c786da6239 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:12.848044 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e9e5b-7b87-48f6-90b4-d2c786da6239 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:12.849652 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e9e5b-7b87-48f6-90b4-d2c786da6239 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:12.860504 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1400/2758] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:17.183393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b31b2-e400-469d-b64a-5238d9666bc0 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:17.262728 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b31b2-e400-469d-b64a-5238d9666bc0 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:17.267654 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:17.268349 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:17.271952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:17.271952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] PATH_INFO: `/v3/users/c42af0abf5054019b8924ae08b892b17` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:17.271952 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c42af0abf5054019b8924ae08b892b17)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:17.272526 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:17.478089 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:17.492023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c42af0abf5054019b8924ae08b892b17'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:17.492813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c42af0abf5054019b8924ae08b892b17'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:17.507623 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c42af0abf5054019b8924ae08b892b17'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:17.563063 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c42af0abf5054019b8924ae08b892b17'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:17.564277 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c42af0abf5054019b8924ae08b892b17'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:17.587079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1359/2759] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:17 2022] DELETE /identity/v3/users/c42af0abf5054019b8924ae08b892b17 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:17.591836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e38ea8-6ef8-4a6c-b66c-2e6f435deeb5 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:17.592358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e38ea8-6ef8-4a6c-b66c-2e6f435deeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:17.592358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e38ea8-6ef8-4a6c-b66c-2e6f435deeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:17.617396 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-50e38ea8-6ef8-4a6c-b66c-2e6f435deeb5 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:17.619070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-50e38ea8-6ef8-4a6c-b66c-2e6f435deeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:17.684468 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1401/2760] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:17.695786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:17.698017 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:17.760422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5b1ff-aecc-45cc-b11e-610e7b1e6c8f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:17.765150 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a04b69-6252-4be1-99dc-7e216dea6424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:17.765762 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04b69-6252-4be1-99dc-7e216dea6424 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:17.766036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04b69-6252-4be1-99dc-7e216dea6424 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:17.766380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04b69-6252-4be1-99dc-7e216dea6424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:17.776241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a04b69-6252-4be1-99dc-7e216dea6424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:17.777701 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a04b69-6252-4be1-99dc-7e216dea6424 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:17.787422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1360/2761] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:18.335183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e38ea8-6ef8-4a6c-b66c-2e6f435deeb5 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:18.337339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-50e38ea8-6ef8-4a6c-b66c-2e6f435deeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:18.349969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e38ea8-6ef8-4a6c-b66c-2e6f435deeb5 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:18.354782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:18.355416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:18.355691 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:18.355988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] PATH_INFO: `/v3/projects/9b6f212db63f480fa2a4c13e47f6a5bb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:18.357763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6f212db63f480fa2a4c13e47f6a5bb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:18.360729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:18.532273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6f212db63f480fa2a4c13e47f6a5bb', 'request_id': 'req-263ad707-f567-488e-bfb3-882c975e7243'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:18.532273 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6f212db63f480fa2a4c13e47f6a5bb', 'request_id': 'req-263ad707-f567-488e-bfb3-882c975e7243'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:18.607496 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6f212db63f480fa2a4c13e47f6a5bb', 'request_id': 'req-263ad707-f567-488e-bfb3-882c975e7243'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:18.625242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] The token cache is being invalidate because project 9b6f212db63f480fa2a4c13e47f6a5bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:18.625566 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:18.626257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:18.628360 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1402/2762] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:18 2022] DELETE /identity/v3/projects/9b6f212db63f480fa2a4c13e47f6a5bb => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:18.632523 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a04b69-6252-4be1-99dc-7e216dea6424 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:18.645965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a04b69-6252-4be1-99dc-7e216dea6424 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:18.762487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:18.762487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:18.762487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:18.762487 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] PATH_INFO: `/v3/users/53d8737b398441c4bcffb9703eb43e5b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:18.763522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d8737b398441c4bcffb9703eb43e5b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:18.763522 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:19.091772 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d8737b398441c4bcffb9703eb43e5b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:19.092764 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d8737b398441c4bcffb9703eb43e5b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:19.098414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:19.106334 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d8737b398441c4bcffb9703eb43e5b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:19.208793 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d8737b398441c4bcffb9703eb43e5b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:19.209512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-06e6c4d0-eadf-4b61-9ab5-d855ff017505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d8737b398441c4bcffb9703eb43e5b'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:19.225179 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1361/2763] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:18 2022] DELETE /identity/v3/users/53d8737b398441c4bcffb9703eb43e5b => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:19.632777 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:19.634637 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:19.657718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:19.664103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ad707-f567-488e-bfb3-882c975e7243 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:19.721407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:19.722069 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:19.722392 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:19.722667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] PATH_INFO: `/v3/projects/a5ae453fd20047daa63436fa1450f766` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:19.723856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5ae453fd20047daa63436fa1450f766)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:19.725831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:19.859272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5ae453fd20047daa63436fa1450f766', 'request_id': 'req-57f89d2d-e160-42e7-be9e-dd815578f003'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:19.866050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5ae453fd20047daa63436fa1450f766', 'request_id': 'req-57f89d2d-e160-42e7-be9e-dd815578f003'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:19.956556 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5ae453fd20047daa63436fa1450f766', 'request_id': 'req-57f89d2d-e160-42e7-be9e-dd815578f003'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:19.967725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] The token cache is being invalidate because project a5ae453fd20047daa63436fa1450f766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:19.968016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:19.968647 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-57f89d2d-e160-42e7-be9e-dd815578f003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:19.970133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1403/2764] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:19 2022] DELETE /identity/v3/projects/a5ae453fd20047daa63436fa1450f766 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:19.975144 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34070628-e5b0-44b3-950b-4b633f6404d2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:19.975616 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34070628-e5b0-44b3-950b-4b633f6404d2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:19.975761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34070628-e5b0-44b3-950b-4b633f6404d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:19.995465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-34070628-e5b0-44b3-950b-4b633f6404d2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:19.996657 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-34070628-e5b0-44b3-950b-4b633f6404d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:20.058550 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1362/2765] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:20.066547 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983086f3-50a3-4156-bafe-cc7fb06d8d2b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:20.066836 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983086f3-50a3-4156-bafe-cc7fb06d8d2b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:20.067358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983086f3-50a3-4156-bafe-cc7fb06d8d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:20.087502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-983086f3-50a3-4156-bafe-cc7fb06d8d2b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:20.088713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-983086f3-50a3-4156-bafe-cc7fb06d8d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:20.096814 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1404/2766] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:20.101480 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34070628-e5b0-44b3-950b-4b633f6404d2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:20.115308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34070628-e5b0-44b3-950b-4b633f6404d2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:20.116335 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a3053-bd54-4c27-b81b-ba98dc93aa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:20.116929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a3053-bd54-4c27-b81b-ba98dc93aa13 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:20.117203 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a3053-bd54-4c27-b81b-ba98dc93aa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:20.117475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a3053-bd54-4c27-b81b-ba98dc93aa13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:20.118208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a3053-bd54-4c27-b81b-ba98dc93aa13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:51:20.118512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a3053-bd54-4c27-b81b-ba98dc93aa13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:20.120084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a3053-bd54-4c27-b81b-ba98dc93aa13 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:20.131311 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1363/2767] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:51:20.135595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983086f3-50a3-4156-bafe-cc7fb06d8d2b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:20.144977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983086f3-50a3-4156-bafe-cc7fb06d8d2b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:20.149907 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8746b60-587b-426e-aa55-e42278da737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:20.150532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8746b60-587b-426e-aa55-e42278da737e admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:20.150812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8746b60-587b-426e-aa55-e42278da737e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:20.151084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8746b60-587b-426e-aa55-e42278da737e admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:20.151715 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8746b60-587b-426e-aa55-e42278da737e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:20.153609 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8746b60-587b-426e-aa55-e42278da737e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:20.249961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1405/2768] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:20 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:20.258878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a3053-bd54-4c27-b81b-ba98dc93aa13 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:20.268865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a3053-bd54-4c27-b81b-ba98dc93aa13 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:20.328879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b32690-e1b8-4b67-b6ef-8bc49d39bf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:20.329542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b32690-e1b8-4b67-b6ef-8bc49d39bf83 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:20.329922 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b32690-e1b8-4b67-b6ef-8bc49d39bf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:20.331171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b32690-e1b8-4b67-b6ef-8bc49d39bf83 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:20.332041 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b32690-e1b8-4b67-b6ef-8bc49d39bf83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:20.334688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b32690-e1b8-4b67-b6ef-8bc49d39bf83 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:20.422804 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1364/2769] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:20 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:51:20.424068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8746b60-587b-426e-aa55-e42278da737e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:20.435224 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8746b60-587b-426e-aa55-e42278da737e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:20.439993 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf64a1-a662-4eca-b2b8-84eebb8af41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:20.440590 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf64a1-a662-4eca-b2b8-84eebb8af41e admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:20.440926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf64a1-a662-4eca-b2b8-84eebb8af41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:20.441233 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf64a1-a662-4eca-b2b8-84eebb8af41e admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:20.441887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf64a1-a662-4eca-b2b8-84eebb8af41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:20.443763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf64a1-a662-4eca-b2b8-84eebb8af41e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:20.456998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1406/2770] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:20.460103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b32690-e1b8-4b67-b6ef-8bc49d39bf83 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:20.470308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b32690-e1b8-4b67-b6ef-8bc49d39bf83 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:20.475845 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b15456-ed8f-4c5b-af4e-29cb582606a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:20.476443 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b15456-ed8f-4c5b-af4e-29cb582606a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:20.476724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b15456-ed8f-4c5b-af4e-29cb582606a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:20.477027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b15456-ed8f-4c5b-af4e-29cb582606a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:20.477675 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b15456-ed8f-4c5b-af4e-29cb582606a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:20.479269 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b15456-ed8f-4c5b-af4e-29cb582606a4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:20.489778 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1365/2771] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:20.493695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf64a1-a662-4eca-b2b8-84eebb8af41e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:20.503680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf64a1-a662-4eca-b2b8-84eebb8af41e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:20.508348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e10f26-7ddb-4bc5-b7d4-6dc847909ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:20.509029 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10f26-7ddb-4bc5-b7d4-6dc847909ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:20.509311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10f26-7ddb-4bc5-b7d4-6dc847909ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:20.509585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10f26-7ddb-4bc5-b7d4-6dc847909ed9 admin admin] PATH_INFO: `/v3/projects/fdcd39c77f8d4378bdc8d6149ee3779b/users/41860290eb484abd96c9f8e33cecc180/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:20.524502 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e10f26-7ddb-4bc5-b7d4-6dc847909ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdcd39c77f8d4378bdc8d6149ee3779b, user_id=41860290eb484abd96c9f8e33cecc180, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:20.527250 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e10f26-7ddb-4bc5-b7d4-6dc847909ed9 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:20.590962 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1407/2772] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:20 2022] PUT /identity/v3/projects/fdcd39c77f8d4378bdc8d6149ee3779b/users/41860290eb484abd96c9f8e33cecc180/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:20.656946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2a913-66ae-4ce5-a607-6cdfe302d423 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:20.657165 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2a913-66ae-4ce5-a607-6cdfe302d423 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:20.657404 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2a913-66ae-4ce5-a607-6cdfe302d423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:20.685970 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-7de2a913-66ae-4ce5-a607-6cdfe302d423 None None] MFA Rules not processed for user `41860290eb484abd96c9f8e33cecc180`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:20.687417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7de2a913-66ae-4ce5-a607-6cdfe302d423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:20.714826 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:20.754242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1366/2773] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:20.765529 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e10f26-7ddb-4bc5-b7d4-6dc847909ed9 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:20.767312 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-a1e10f26-7ddb-4bc5-b7d4-6dc847909ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:20.798960 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:20.824028 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e10f26-7ddb-4bc5-b7d4-6dc847909ed9 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:20.827686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c4f95-e917-4147-b0bc-693027c722fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:20.828285 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4f95-e917-4147-b0bc-693027c722fc service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:20.828563 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4f95-e917-4147-b0bc-693027c722fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:20.828864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4f95-e917-4147-b0bc-693027c722fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:20.837968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c4f95-e917-4147-b0bc-693027c722fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:20.839429 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c4f95-e917-4147-b0bc-693027c722fc service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:20.848270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1408/2774] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:21.580438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2a913-66ae-4ce5-a607-6cdfe302d423 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:21.581954 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-7de2a913-66ae-4ce5-a607-6cdfe302d423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:21.651730 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2a913-66ae-4ce5-a607-6cdfe302d423 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:21.659325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:21.659666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:21.659951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:21.660230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] PATH_INFO: `/v3/users/33e51bb597844cea82394afaad18eaaa` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:21.662916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e51bb597844cea82394afaad18eaaa)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:21.665005 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:21.832890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e51bb597844cea82394afaad18eaaa'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:21.833559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e51bb597844cea82394afaad18eaaa'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:21.845857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e51bb597844cea82394afaad18eaaa'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:21.917213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e51bb597844cea82394afaad18eaaa'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:21.917848 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e51bb597844cea82394afaad18eaaa'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:21.931315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1367/2775] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:21 2022] DELETE /identity/v3/users/33e51bb597844cea82394afaad18eaaa => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:21.935855 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a29374-c941-4ffd-a7e0-ffcc7657a064 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:21.936317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a29374-c941-4ffd-a7e0-ffcc7657a064 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:21.936430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a29374-c941-4ffd-a7e0-ffcc7657a064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:21.955625 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-e9a29374-c941-4ffd-a7e0-ffcc7657a064 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:21.956834 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e9a29374-c941-4ffd-a7e0-ffcc7657a064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:22.015863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1409/2776] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:22.026050 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:22.027465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:22.079080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:22.084756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7724469-cd9f-48db-91e6-b658cf19c30e admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:22.088518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fd9f7-1df2-4f85-8588-6fafc81dec83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:22.090614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fd9f7-1df2-4f85-8588-6fafc81dec83 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:22.090614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fd9f7-1df2-4f85-8588-6fafc81dec83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:22.090614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fd9f7-1df2-4f85-8588-6fafc81dec83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:22.098676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fd9f7-1df2-4f85-8588-6fafc81dec83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:22.100744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fd9f7-1df2-4f85-8588-6fafc81dec83 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:22.110156 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1368/2777] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:23.010835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a29374-c941-4ffd-a7e0-ffcc7657a064 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:23.012358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e9a29374-c941-4ffd-a7e0-ffcc7657a064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:23.050112 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a29374-c941-4ffd-a7e0-ffcc7657a064 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:23.060310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:23.060979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:23.061264 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:23.061539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] PATH_INFO: `/v3/projects/bb52515b3bb846b39ca83b99744c5692` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:23.062756 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb52515b3bb846b39ca83b99744c5692)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:23.064705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:23.168721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb52515b3bb846b39ca83b99744c5692', 'request_id': 'req-e598e78b-6b73-42ec-8431-b553b354f37a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:23.175046 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb52515b3bb846b39ca83b99744c5692', 'request_id': 'req-e598e78b-6b73-42ec-8431-b553b354f37a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:23.227296 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb52515b3bb846b39ca83b99744c5692', 'request_id': 'req-e598e78b-6b73-42ec-8431-b553b354f37a'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:23.241564 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] The token cache is being invalidate because project bb52515b3bb846b39ca83b99744c5692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:23.242059 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:23.242725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:23.244314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1410/2778] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:23 2022] DELETE /identity/v3/projects/bb52515b3bb846b39ca83b99744c5692 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:23.249786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fd9f7-1df2-4f85-8588-6fafc81dec83 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:23.259910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fd9f7-1df2-4f85-8588-6fafc81dec83 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:23.322719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:23.323352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:23.323627 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:23.323896 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] PATH_INFO: `/v3/users/7253ebb304ed4c60a3da0983285875c7` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:23.325108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7253ebb304ed4c60a3da0983285875c7)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:23.327437 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:23.588449 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7253ebb304ed4c60a3da0983285875c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:23.589094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7253ebb304ed4c60a3da0983285875c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:23.602204 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7253ebb304ed4c60a3da0983285875c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:23.671490 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7253ebb304ed4c60a3da0983285875c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:23.672219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-3579db63-420b-4c04-84cb-78c742ba4f8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7253ebb304ed4c60a3da0983285875c7'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:23.685828 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1369/2779] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:23 2022] DELETE /identity/v3/users/7253ebb304ed4c60a3da0983285875c7 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:24.266943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:24.268469 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:24.283113 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e78b-6b73-42ec-8431-b553b354f37a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:24.335475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:24.342323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:24.342947 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:24.343222 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:24.343533 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] PATH_INFO: `/v3/projects/d498116cdb0d485291403b1d8e33f9fa` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:24.344641 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d498116cdb0d485291403b1d8e33f9fa)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:24.346575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:24.437501 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd498116cdb0d485291403b1d8e33f9fa', 'request_id': 'req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:24.444636 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd498116cdb0d485291403b1d8e33f9fa', 'request_id': 'req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:24.527643 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd498116cdb0d485291403b1d8e33f9fa', 'request_id': 'req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:24.538295 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] The token cache is being invalidate because project d498116cdb0d485291403b1d8e33f9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:24.538584 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:24.539248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-0ec07189-82b6-46f7-808c-c1a2d5f08ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:24.540739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1411/2780] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:24 2022] DELETE /identity/v3/projects/d498116cdb0d485291403b1d8e33f9fa => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:24.545813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10e502-fe6b-43bf-b9a7-a5ea70136339 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:24.546306 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10e502-fe6b-43bf-b9a7-a5ea70136339 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:24.546424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10e502-fe6b-43bf-b9a7-a5ea70136339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:24.568491 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-3d10e502-fe6b-43bf-b9a7-a5ea70136339 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:24.569709 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-3d10e502-fe6b-43bf-b9a7-a5ea70136339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:24.630103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1370/2781] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:51:24.637969 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b1d10-22d4-4dab-9483-7aa2d033533b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:24.638291 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b1d10-22d4-4dab-9483-7aa2d033533b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:24.638571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b1d10-22d4-4dab-9483-7aa2d033533b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:24.658762 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-490b1d10-22d4-4dab-9483-7aa2d033533b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:24.659966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-490b1d10-22d4-4dab-9483-7aa2d033533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:24.668361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1412/2782] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:24.672733 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10e502-fe6b-43bf-b9a7-a5ea70136339 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:24.682846 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10e502-fe6b-43bf-b9a7-a5ea70136339 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:24.687424 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e922-c89a-4788-b808-3dfcb40aa61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:24.690878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e922-c89a-4788-b808-3dfcb40aa61f admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:24.690878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e922-c89a-4788-b808-3dfcb40aa61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:24.690878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e922-c89a-4788-b808-3dfcb40aa61f admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:24.690878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9e922-c89a-4788-b808-3dfcb40aa61f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:51:24.690878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9e922-c89a-4788-b808-3dfcb40aa61f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:24.691377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9e922-c89a-4788-b808-3dfcb40aa61f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:24.693631 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:24.702783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1371/2783] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:51:24.707012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b1d10-22d4-4dab-9483-7aa2d033533b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:24.716456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b1d10-22d4-4dab-9483-7aa2d033533b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:24.721355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8c3b7-5de3-4f03-8eb1-7a9714466cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:24.721946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8c3b7-5de3-4f03-8eb1-7a9714466cac admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:24.722270 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8c3b7-5de3-4f03-8eb1-7a9714466cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:24.722544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8c3b7-5de3-4f03-8eb1-7a9714466cac admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:24.723198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8c3b7-5de3-4f03-8eb1-7a9714466cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:24.725594 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8c3b7-5de3-4f03-8eb1-7a9714466cac admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:24.815546 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1413/2784] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:24 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:24.819703 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e922-c89a-4788-b808-3dfcb40aa61f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:24.829867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e922-c89a-4788-b808-3dfcb40aa61f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:24.888114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cfcd2-0ca3-4dfb-b249-c0e81a0457ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:24.888720 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cfcd2-0ca3-4dfb-b249-c0e81a0457ab admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:24.888998 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cfcd2-0ca3-4dfb-b249-c0e81a0457ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:24.889267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cfcd2-0ca3-4dfb-b249-c0e81a0457ab admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:24.889893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cfcd2-0ca3-4dfb-b249-c0e81a0457ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:24.894230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cfcd2-0ca3-4dfb-b249-c0e81a0457ab admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:24.975655 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1372/2785] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:24 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:51:24.981870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8c3b7-5de3-4f03-8eb1-7a9714466cac admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:24.992303 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8c3b7-5de3-4f03-8eb1-7a9714466cac admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:24.997064 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b64a3-792f-404a-ad5d-ffcc0cc3b77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:24.997657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b64a3-792f-404a-ad5d-ffcc0cc3b77d admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.001946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b64a3-792f-404a-ad5d-ffcc0cc3b77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.001946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b64a3-792f-404a-ad5d-ffcc0cc3b77d admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.001946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b64a3-792f-404a-ad5d-ffcc0cc3b77d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:25.001946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b64a3-792f-404a-ad5d-ffcc0cc3b77d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:25.010343 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1414/2786] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:25.014557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cfcd2-0ca3-4dfb-b249-c0e81a0457ab admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:25.025211 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cfcd2-0ca3-4dfb-b249-c0e81a0457ab admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:25.029804 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320bff8-a9df-4ac5-a568-3f1831dc4914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:25.030438 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320bff8-a9df-4ac5-a568-3f1831dc4914 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.030719 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320bff8-a9df-4ac5-a568-3f1831dc4914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.030994 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320bff8-a9df-4ac5-a568-3f1831dc4914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.031626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320bff8-a9df-4ac5-a568-3f1831dc4914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:25.033197 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320bff8-a9df-4ac5-a568-3f1831dc4914 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:25.045549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1373/2787] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:25.049697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b64a3-792f-404a-ad5d-ffcc0cc3b77d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:25.059565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b64a3-792f-404a-ad5d-ffcc0cc3b77d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:25.064278 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9228bd4-5367-4c63-8a29-95c3e033372c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:25.064892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9228bd4-5367-4c63-8a29-95c3e033372c admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.065165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9228bd4-5367-4c63-8a29-95c3e033372c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.065436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9228bd4-5367-4c63-8a29-95c3e033372c admin admin] PATH_INFO: `/v3/projects/b51b73ebeb994f92b67d426bc5babc71/users/c46bd005feec40e8a6bc68580c29e32c/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.080202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9228bd4-5367-4c63-8a29-95c3e033372c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b51b73ebeb994f92b67d426bc5babc71, user_id=c46bd005feec40e8a6bc68580c29e32c, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:25.082862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9228bd4-5367-4c63-8a29-95c3e033372c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:25.163133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1415/2788] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:25 2022] PUT /identity/v3/projects/b51b73ebeb994f92b67d426bc5babc71/users/c46bd005feec40e8a6bc68580c29e32c/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:25.173606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e97af-7491-42e9-88f1-692a6ddd20c4 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.173891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e97af-7491-42e9-88f1-692a6ddd20c4 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.174188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e97af-7491-42e9-88f1-692a6ddd20c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.202519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-415e97af-7491-42e9-88f1-692a6ddd20c4 None None] MFA Rules not processed for user `c46bd005feec40e8a6bc68580c29e32c`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:25.204405 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-415e97af-7491-42e9-88f1-692a6ddd20c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:25.283931 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1374/2789] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:51:25.289084 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9228bd4-5367-4c63-8a29-95c3e033372c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:25.354761 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9228bd4-5367-4c63-8a29-95c3e033372c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:25.359631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1ef4-1c32-40f8-aba7-b4fd10a3cf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:25.360483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1ef4-1c32-40f8-aba7-b4fd10a3cf6b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.360688 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1ef4-1c32-40f8-aba7-b4fd10a3cf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.360918 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1ef4-1c32-40f8-aba7-b4fd10a3cf6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.361549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb1ef4-1c32-40f8-aba7-b4fd10a3cf6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:25.363479 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb1ef4-1c32-40f8-aba7-b4fd10a3cf6b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:25.483573 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1416/2790] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:25 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:25.487408 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e97af-7491-42e9-88f1-692a6ddd20c4 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:25.498081 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e97af-7491-42e9-88f1-692a6ddd20c4 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:25.555079 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3e54b-4129-4984-85b6-9c1770055b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:25.555585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3e54b-4129-4984-85b6-9c1770055b45 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.560384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3e54b-4129-4984-85b6-9c1770055b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.560384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3e54b-4129-4984-85b6-9c1770055b45 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.560384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3e54b-4129-4984-85b6-9c1770055b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:25.560384 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3e54b-4129-4984-85b6-9c1770055b45 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:25.569635 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:25.642609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1375/2791] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:25 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:25.647316 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1ef4-1c32-40f8-aba7-b4fd10a3cf6b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:25.657678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1ef4-1c32-40f8-aba7-b4fd10a3cf6b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:25.662308 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0524ff-0daa-474e-8369-8642db7911ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:25.662921 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0524ff-0daa-474e-8369-8642db7911ac admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.663197 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0524ff-0daa-474e-8369-8642db7911ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.663467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0524ff-0daa-474e-8369-8642db7911ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.664111 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0524ff-0daa-474e-8369-8642db7911ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:25.665718 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0524ff-0daa-474e-8369-8642db7911ac admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:25.666988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:25.677686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1417/2792] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:25.683049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3e54b-4129-4984-85b6-9c1770055b45 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:25.693807 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3e54b-4129-4984-85b6-9c1770055b45 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:25.698409 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252bbfd-c82a-494a-a507-94a231bda50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:25.699052 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252bbfd-c82a-494a-a507-94a231bda50d admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.699328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252bbfd-c82a-494a-a507-94a231bda50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.699597 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252bbfd-c82a-494a-a507-94a231bda50d admin admin] PATH_INFO: `/v3/projects/659c26a1d2b14f4ab91c5853e4fdeaf9/users/ccbb579d9ee046eaadbceead94df882c/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.715606 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252bbfd-c82a-494a-a507-94a231bda50d admin admin] RBAC: Authorizing `identity:create_grant(project_id=659c26a1d2b14f4ab91c5853e4fdeaf9, user_id=ccbb579d9ee046eaadbceead94df882c, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:25.718289 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252bbfd-c82a-494a-a507-94a231bda50d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:25.791584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1376/2793] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:25 2022] PUT /identity/v3/projects/659c26a1d2b14f4ab91c5853e4fdeaf9/users/ccbb579d9ee046eaadbceead94df882c/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:25.795570 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0524ff-0daa-474e-8369-8642db7911ac admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:25.805806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0524ff-0daa-474e-8369-8642db7911ac admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:25.861074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9c4b9-2e6d-43e5-b09a-50860e7fc03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:25.861678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c4b9-2e6d-43e5-b09a-50860e7fc03b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.861959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c4b9-2e6d-43e5-b09a-50860e7fc03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.862292 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c4b9-2e6d-43e5-b09a-50860e7fc03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.862926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9c4b9-2e6d-43e5-b09a-50860e7fc03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:25.865058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9c4b9-2e6d-43e5-b09a-50860e7fc03b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:25.874539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1418/2794] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:25.878713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252bbfd-c82a-494a-a507-94a231bda50d admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:25.890175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252bbfd-c82a-494a-a507-94a231bda50d admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:25.895042 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717aeb2-a9ab-490f-bd3e-9b23bf1e91e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:25.895836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717aeb2-a9ab-490f-bd3e-9b23bf1e91e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.896114 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717aeb2-a9ab-490f-bd3e-9b23bf1e91e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.896413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717aeb2-a9ab-490f-bd3e-9b23bf1e91e2 admin admin] PATH_INFO: `/v3/projects/659c26a1d2b14f4ab91c5853e4fdeaf9/users/ccbb579d9ee046eaadbceead94df882c/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.898474 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717aeb2-a9ab-490f-bd3e-9b23bf1e91e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=659c26a1d2b14f4ab91c5853e4fdeaf9, user_id=ccbb579d9ee046eaadbceead94df882c, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:25.901073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717aeb2-a9ab-490f-bd3e-9b23bf1e91e2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:25.911744 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1377/2795] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:25 2022] PUT /identity/v3/projects/659c26a1d2b14f4ab91c5853e4fdeaf9/users/ccbb579d9ee046eaadbceead94df882c/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:25.923118 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa89b-4f1a-4f10-90b2-252b529c0edc None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:25.923402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa89b-4f1a-4f10-90b2-252b529c0edc None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:25.923679 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa89b-4f1a-4f10-90b2-252b529c0edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:25.953780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f73aa89b-4f1a-4f10-90b2-252b529c0edc None None] MFA Rules not processed for user `ccbb579d9ee046eaadbceead94df882c`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:25.955036 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f73aa89b-4f1a-4f10-90b2-252b529c0edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:26.027578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1419/2796] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:26.038323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717aeb2-a9ab-490f-bd3e-9b23bf1e91e2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:26.040030 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f717aeb2-a9ab-490f-bd3e-9b23bf1e91e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:26.097259 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717aeb2-a9ab-490f-bd3e-9b23bf1e91e2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:26.101589 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf91563-1eb0-4377-bb33-51a46cb0f563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:26.102231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf91563-1eb0-4377-bb33-51a46cb0f563 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:26.102512 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf91563-1eb0-4377-bb33-51a46cb0f563 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:26.102781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf91563-1eb0-4377-bb33-51a46cb0f563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:26.111888 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf91563-1eb0-4377-bb33-51a46cb0f563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:26.115482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf91563-1eb0-4377-bb33-51a46cb0f563 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:26.123322 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1378/2797] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:27.964173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73aa89b-4f1a-4f10-90b2-252b529c0edc None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:28.034664 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73aa89b-4f1a-4f10-90b2-252b529c0edc None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:28.039994 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:28.040741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:28.041158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:28.041494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] PATH_INFO: `/v3/users/c46bd005feec40e8a6bc68580c29e32c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:28.042714 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c46bd005feec40e8a6bc68580c29e32c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:28.047883 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:28.066300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:28.200346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c46bd005feec40e8a6bc68580c29e32c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:28.217721 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c46bd005feec40e8a6bc68580c29e32c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:28.218473 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c46bd005feec40e8a6bc68580c29e32c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:28.299575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c46bd005feec40e8a6bc68580c29e32c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:28.300220 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c46bd005feec40e8a6bc68580c29e32c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:28.316935 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1420/2798] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:27 2022] DELETE /identity/v3/users/c46bd005feec40e8a6bc68580c29e32c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:51:28.319220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74035f48-5611-4a53-a19d-7866a364fef8 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:28.319726 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74035f48-5611-4a53-a19d-7866a364fef8 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:28.319943 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74035f48-5611-4a53-a19d-7866a364fef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:28.341798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-74035f48-5611-4a53-a19d-7866a364fef8 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:28.343147 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-74035f48-5611-4a53-a19d-7866a364fef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:28.409208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1379/2799] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:28.419357 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:28.420797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:28.477416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d67993-1137-475d-896e-0c63321e6061 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:28.480986 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0156123-89eb-4586-8ec6-470d25530803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:28.481604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0156123-89eb-4586-8ec6-470d25530803 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:28.481879 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0156123-89eb-4586-8ec6-470d25530803 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:28.482186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0156123-89eb-4586-8ec6-470d25530803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:28.491181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0156123-89eb-4586-8ec6-470d25530803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:28.492595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0156123-89eb-4586-8ec6-470d25530803 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:28.502007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1421/2800] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:28.936381 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74035f48-5611-4a53-a19d-7866a364fef8 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:28.937951 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-74035f48-5611-4a53-a19d-7866a364fef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:28.951006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74035f48-5611-4a53-a19d-7866a364fef8 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:28.963137 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:28.963781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:28.964104 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:28.964320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] PATH_INFO: `/v3/projects/b51b73ebeb994f92b67d426bc5babc71` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:28.965564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b51b73ebeb994f92b67d426bc5babc71)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:28.967964 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:29.089946 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b51b73ebeb994f92b67d426bc5babc71', 'request_id': 'req-00702e1e-748b-43a5-8e5b-9a636088f9d3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:29.096426 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b51b73ebeb994f92b67d426bc5babc71', 'request_id': 'req-00702e1e-748b-43a5-8e5b-9a636088f9d3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:29.098088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:29.167665 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b51b73ebeb994f92b67d426bc5babc71', 'request_id': 'req-00702e1e-748b-43a5-8e5b-9a636088f9d3'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:29.179202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] The token cache is being invalidate because project b51b73ebeb994f92b67d426bc5babc71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:29.179493 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:29.180131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:29.181624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1380/2801] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:28 2022] DELETE /identity/v3/projects/b51b73ebeb994f92b67d426bc5babc71 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:29.185782 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0156123-89eb-4586-8ec6-470d25530803 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:29.195670 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0156123-89eb-4586-8ec6-470d25530803 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:29.256877 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:29.257415 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:29.257742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:29.258020 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] PATH_INFO: `/v3/users/ccbb579d9ee046eaadbceead94df882c` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:29.259190 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccbb579d9ee046eaadbceead94df882c)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:29.261140 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:29.439729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:29.445268 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccbb579d9ee046eaadbceead94df882c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:29.457900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccbb579d9ee046eaadbceead94df882c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:29.459242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccbb579d9ee046eaadbceead94df882c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:29.585229 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccbb579d9ee046eaadbceead94df882c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:29.585873 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4611e3a7-a2cc-4bff-92a4-4545d1c90e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccbb579d9ee046eaadbceead94df882c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:29.598722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1422/2802] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:29 2022] DELETE /identity/v3/users/ccbb579d9ee046eaadbceead94df882c => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:29.986105 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:29.987629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:30.001918 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00702e1e-748b-43a5-8e5b-9a636088f9d3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:30.059686 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:30.060316 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:30.060593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:30.060866 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] PATH_INFO: `/v3/projects/659c26a1d2b14f4ab91c5853e4fdeaf9` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:30.061960 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=659c26a1d2b14f4ab91c5853e4fdeaf9)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:30.063879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:30.162077 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659c26a1d2b14f4ab91c5853e4fdeaf9', 'request_id': 'req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:30.168318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659c26a1d2b14f4ab91c5853e4fdeaf9', 'request_id': 'req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:30.235955 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659c26a1d2b14f4ab91c5853e4fdeaf9', 'request_id': 'req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:30.249636 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] The token cache is being invalidate because project 659c26a1d2b14f4ab91c5853e4fdeaf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:30.249927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:30.250587 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-0164b8f8-f05f-4e7f-b446-a1fbfc4b08e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:30.252161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1381/2803] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:29 2022] DELETE /identity/v3/projects/659c26a1d2b14f4ab91c5853e4fdeaf9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:30.257651 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4d426-e08f-494f-ade8-945d2bbfd755 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:30.258108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4d426-e08f-494f-ade8-945d2bbfd755 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:30.258251 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4d426-e08f-494f-ade8-945d2bbfd755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:30.277682 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b9f4d426-e08f-494f-ade8-945d2bbfd755 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:30.278863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b9f4d426-e08f-494f-ade8-945d2bbfd755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:30.337007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1423/2804] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:30.344708 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b194847-cbaa-412e-9a1d-073befaecede None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:30.344997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b194847-cbaa-412e-9a1d-073befaecede None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:30.345274 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b194847-cbaa-412e-9a1d-073befaecede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:30.366064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-5b194847-cbaa-412e-9a1d-073befaecede None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:30.367326 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5b194847-cbaa-412e-9a1d-073befaecede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:30.375208 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1382/2805] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:30.379275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4d426-e08f-494f-ade8-945d2bbfd755 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:30.389393 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4d426-e08f-494f-ade8-945d2bbfd755 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:30.393980 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7aa4c-4db3-40a9-afe7-da79df73a88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:30.394610 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7aa4c-4db3-40a9-afe7-da79df73a88a admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:30.394894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7aa4c-4db3-40a9-afe7-da79df73a88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:30.395171 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7aa4c-4db3-40a9-afe7-da79df73a88a admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:30.395839 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7aa4c-4db3-40a9-afe7-da79df73a88a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:51:30.396173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7aa4c-4db3-40a9-afe7-da79df73a88a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:30.397773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7aa4c-4db3-40a9-afe7-da79df73a88a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:30.408934 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1424/2806] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:51:30.413220 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b194847-cbaa-412e-9a1d-073befaecede None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:30.422267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b194847-cbaa-412e-9a1d-073befaecede None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:30.427377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585ff91-a35f-465b-8a86-9385026091b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:30.427958 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585ff91-a35f-465b-8a86-9385026091b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:30.428233 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585ff91-a35f-465b-8a86-9385026091b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:30.428499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585ff91-a35f-465b-8a86-9385026091b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:30.429118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585ff91-a35f-465b-8a86-9385026091b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:30.431014 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585ff91-a35f-465b-8a86-9385026091b3 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:30.517173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1383/2807] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:30 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:51:30.521297 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7aa4c-4db3-40a9-afe7-da79df73a88a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:30.536694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7aa4c-4db3-40a9-afe7-da79df73a88a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:30.599481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da582ce5-2536-410b-a990-55b1ebb4a6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:30.600176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da582ce5-2536-410b-a990-55b1ebb4a6ea admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:30.600523 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da582ce5-2536-410b-a990-55b1ebb4a6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:30.600976 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da582ce5-2536-410b-a990-55b1ebb4a6ea admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:30.601780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da582ce5-2536-410b-a990-55b1ebb4a6ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:30.603791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da582ce5-2536-410b-a990-55b1ebb4a6ea admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:30.691734 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1425/2808] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:30 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:30.704927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585ff91-a35f-465b-8a86-9385026091b3 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:30.704927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585ff91-a35f-465b-8a86-9385026091b3 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:30.712308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e804f9-6595-4374-a1d9-d3b881dc94e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:30.713270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e804f9-6595-4374-a1d9-d3b881dc94e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:30.713605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e804f9-6595-4374-a1d9-d3b881dc94e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:30.713879 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e804f9-6595-4374-a1d9-d3b881dc94e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:30.714972 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e804f9-6595-4374-a1d9-d3b881dc94e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:30.716365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e804f9-6595-4374-a1d9-d3b881dc94e7 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:30.731134 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1384/2809] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:51:30.732517 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da582ce5-2536-410b-a990-55b1ebb4a6ea admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:30.751748 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da582ce5-2536-410b-a990-55b1ebb4a6ea admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:30.756657 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a5aa7-3068-43b4-8bfe-126fce4b5138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:30.758596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a5aa7-3068-43b4-8bfe-126fce4b5138 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:30.758596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a5aa7-3068-43b4-8bfe-126fce4b5138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:30.758596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a5aa7-3068-43b4-8bfe-126fce4b5138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:30.758596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a5aa7-3068-43b4-8bfe-126fce4b5138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:30.760201 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a5aa7-3068-43b4-8bfe-126fce4b5138 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:30.770742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1426/2810] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:30.774887 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e804f9-6595-4374-a1d9-d3b881dc94e7 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:30.784975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e804f9-6595-4374-a1d9-d3b881dc94e7 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:30.790316 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830fe27-ce36-4100-9ade-df7422d1be3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:30.790905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830fe27-ce36-4100-9ade-df7422d1be3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:30.791230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830fe27-ce36-4100-9ade-df7422d1be3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:30.791508 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830fe27-ce36-4100-9ade-df7422d1be3c admin admin] PATH_INFO: `/v3/projects/c694c9be27894b609aa2675d8d70d4dd/users/eb6e383c27d04060a72f88b0d5d549f1/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:30.806432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830fe27-ce36-4100-9ade-df7422d1be3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c694c9be27894b609aa2675d8d70d4dd, user_id=eb6e383c27d04060a72f88b0d5d549f1, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:30.809177 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830fe27-ce36-4100-9ade-df7422d1be3c admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:30.899317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1385/2811] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:30 2022] PUT /identity/v3/projects/c694c9be27894b609aa2675d8d70d4dd/users/eb6e383c27d04060a72f88b0d5d549f1/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:30.911794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763161f-1290-43b0-9c78-cf44eb82f508 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:30.912097 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763161f-1290-43b0-9c78-cf44eb82f508 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:30.912404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763161f-1290-43b0-9c78-cf44eb82f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:30.940134 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-6763161f-1290-43b0-9c78-cf44eb82f508 None None] MFA Rules not processed for user `eb6e383c27d04060a72f88b0d5d549f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:30.941365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-6763161f-1290-43b0-9c78-cf44eb82f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:30.953158 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:31.004176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1427/2812] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:31.015777 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830fe27-ce36-4100-9ade-df7422d1be3c admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:31.017489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e830fe27-ce36-4100-9ade-df7422d1be3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:31.028415 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:31.073465 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830fe27-ce36-4100-9ade-df7422d1be3c admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:31.077044 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab210c-9b18-47ae-bb5e-e6bb4a0cce0f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c34eb8fcef0543c282b3887dfeb74c15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:31.077640 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab210c-9b18-47ae-bb5e-e6bb4a0cce0f service glance] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:31.077919 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab210c-9b18-47ae-bb5e-e6bb4a0cce0f service glance] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:31.078218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab210c-9b18-47ae-bb5e-e6bb4a0cce0f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:31.087666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab210c-9b18-47ae-bb5e-e6bb4a0cce0f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:31.089073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab210c-9b18-47ae-bb5e-e6bb4a0cce0f service glance] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:31.097644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1386/2813] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:37.566050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff0f5b-fe68-43f5-9ab2-7dcfaaff86f6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:37.566549 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff0f5b-fe68-43f5-9ab2-7dcfaaff86f6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:37.566697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff0f5b-fe68-43f5-9ab2-7dcfaaff86f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:37.586900 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f3ff0f5b-fe68-43f5-9ab2-7dcfaaff86f6 None None] MFA Rules not processed for user `e04f49a794c5487ba372cf1baaf67402`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:37.588531 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f3ff0f5b-fe68-43f5-9ab2-7dcfaaff86f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:37.650630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1428/2814] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:40.033309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab210c-9b18-47ae-bb5e-e6bb4a0cce0f service glance] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:40.034780 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9bab210c-9b18-47ae-bb5e-e6bb4a0cce0f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:40.102721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab210c-9b18-47ae-bb5e-e6bb4a0cce0f service glance] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:40.107263 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:40.107889 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:40.108168 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:40.108441 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] PATH_INFO: `/v3/users/4ebc957cf725466f92ef2238a0eadf20` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:40.109570 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ebc957cf725466f92ef2238a0eadf20)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:40.111506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:40.274320 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ebc957cf725466f92ef2238a0eadf20'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:40.274869 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ebc957cf725466f92ef2238a0eadf20'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:40.287267 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ebc957cf725466f92ef2238a0eadf20'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:40.367935 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ebc957cf725466f92ef2238a0eadf20'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:40.368557 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ebc957cf725466f92ef2238a0eadf20'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:40.383452 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1387/2815] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:40 2022] DELETE /identity/v3/users/4ebc957cf725466f92ef2238a0eadf20 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:40.388204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9882b-08e4-467c-8d23-980f9e7c4a0b None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:40.388204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9882b-08e4-467c-8d23-980f9e7c4a0b None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:40.388769 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9882b-08e4-467c-8d23-980f9e7c4a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:40.407121 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-b5f9882b-08e4-467c-8d23-980f9e7c4a0b None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:40.408332 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b5f9882b-08e4-467c-8d23-980f9e7c4a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:40.472367 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1429/2816] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:40.489275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:40.491737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:40.550080 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:40.589801 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b2dc6-7674-41d5-a3c7-0d32ac4084bb admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:40.598486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96c33-11f4-4d20-af68-61af61dbd9f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:40.599670 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96c33-11f4-4d20-af68-61af61dbd9f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:40.600985 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96c33-11f4-4d20-af68-61af61dbd9f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:40.601662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a96c33-11f4-4d20-af68-61af61dbd9f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:40.618006 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a96c33-11f4-4d20-af68-61af61dbd9f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:40.620125 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a96c33-11f4-4d20-af68-61af61dbd9f5 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:40.646695 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1388/2817] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:41.235571 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9882b-08e4-467c-8d23-980f9e7c4a0b None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:41.237181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b5f9882b-08e4-467c-8d23-980f9e7c4a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:41.249909 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9882b-08e4-467c-8d23-980f9e7c4a0b None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:41.254831 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:41.256186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:41.256186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:41.257486 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] PATH_INFO: `/v3/projects/55a23449c88b45f68be04bcdcd1e7a98` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:41.258794 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55a23449c88b45f68be04bcdcd1e7a98)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:41.263290 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:41.400538 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55a23449c88b45f68be04bcdcd1e7a98', 'request_id': 'req-84015912-93eb-4dd2-b633-5a36438cdce2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:41.407963 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55a23449c88b45f68be04bcdcd1e7a98', 'request_id': 'req-84015912-93eb-4dd2-b633-5a36438cdce2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:41.409677 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:41.511065 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55a23449c88b45f68be04bcdcd1e7a98', 'request_id': 'req-84015912-93eb-4dd2-b633-5a36438cdce2'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:41.524568 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] The token cache is being invalidate because project 55a23449c88b45f68be04bcdcd1e7a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:41.524709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:41.525314 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-84015912-93eb-4dd2-b633-5a36438cdce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:41.526919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1430/2818] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:41 2022] DELETE /identity/v3/projects/55a23449c88b45f68be04bcdcd1e7a98 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:50.694534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=12 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:51.568237 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=4 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:56.713432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96c33-11f4-4d20-af68-61af61dbd9f5 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:56.715206 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-78a96c33-11f4-4d20-af68-61af61dbd9f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:56.728330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a96c33-11f4-4d20-af68-61af61dbd9f5 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:56.789538 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:56.799339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:56.799339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:56.799339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] PATH_INFO: `/v3/users/eb6e383c27d04060a72f88b0d5d549f1` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:56.799339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb6e383c27d04060a72f88b0d5d549f1)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:56.799339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:56.998248 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb6e383c27d04060a72f88b0d5d549f1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:56.998902 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb6e383c27d04060a72f88b0d5d549f1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:57.011212 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb6e383c27d04060a72f88b0d5d549f1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:57.104891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb6e383c27d04060a72f88b0d5d549f1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:57.105556 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb6e383c27d04060a72f88b0d5d549f1'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:57.119505 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1389/2819] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:51:56 2022] DELETE /identity/v3/users/eb6e383c27d04060a72f88b0d5d549f1 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:57.124023 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56ae3c-61f5-48bd-87df-685e0ebc25bd None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:57.124522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56ae3c-61f5-48bd-87df-685e0ebc25bd None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:57.124629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56ae3c-61f5-48bd-87df-685e0ebc25bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:57.143895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ca56ae3c-61f5-48bd-87df-685e0ebc25bd None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:57.145161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ca56ae3c-61f5-48bd-87df-685e0ebc25bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:57.204452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1431/2820] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:57.214309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:57.215818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:57.272084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da72a49-3293-42ce-95ac-79f891dfb47f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:57.275731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd14ca-e279-4e2c-b2b9-d098b6cfd425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:57.276324 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd14ca-e279-4e2c-b2b9-d098b6cfd425 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:57.276599 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd14ca-e279-4e2c-b2b9-d098b6cfd425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:57.276867 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd14ca-e279-4e2c-b2b9-d098b6cfd425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:57.285865 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd14ca-e279-4e2c-b2b9-d098b6cfd425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:57.287352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd14ca-e279-4e2c-b2b9-d098b6cfd425 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:57.299486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1390/2821] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:51:57.717074 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56ae3c-61f5-48bd-87df-685e0ebc25bd None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:57.718618 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ca56ae3c-61f5-48bd-87df-685e0ebc25bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:57.731240 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56ae3c-61f5-48bd-87df-685e0ebc25bd None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:57.735951 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:57.736585 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:57.736864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:57.737416 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] PATH_INFO: `/v3/projects/c694c9be27894b609aa2675d8d70d4dd` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:57.738443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c694c9be27894b609aa2675d8d70d4dd)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:57.740371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:57.830407 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c694c9be27894b609aa2675d8d70d4dd', 'request_id': 'req-f52aa865-486e-4b83-99cf-6bf4387af059'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:57.836806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c694c9be27894b609aa2675d8d70d4dd', 'request_id': 'req-f52aa865-486e-4b83-99cf-6bf4387af059'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:57.892943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c694c9be27894b609aa2675d8d70d4dd', 'request_id': 'req-f52aa865-486e-4b83-99cf-6bf4387af059'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:57.904435 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] The token cache is being invalidate because project c694c9be27894b609aa2675d8d70d4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:51:57.904724 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:57.905357 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f52aa865-486e-4b83-99cf-6bf4387af059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:51:57.906926 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1432/2822] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:51:57 2022] DELETE /identity/v3/projects/c694c9be27894b609aa2675d8d70d4dd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:57.911813 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398fc50-96ad-4164-b8ae-fe4328839d82 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:57.912276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398fc50-96ad-4164-b8ae-fe4328839d82 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:57.912397 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398fc50-96ad-4164-b8ae-fe4328839d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:57.932022 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d398fc50-96ad-4164-b8ae-fe4328839d82 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:57.933200 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d398fc50-96ad-4164-b8ae-fe4328839d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:57.949566 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:57.990377 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1391/2823] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:57.998301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7671f8-4819-40ec-a353-5add1a20d459 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:57.998592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7671f8-4819-40ec-a353-5add1a20d459 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:57.998876 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7671f8-4819-40ec-a353-5add1a20d459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.018658 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-4d7671f8-4819-40ec-a353-5add1a20d459 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:58.019859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4d7671f8-4819-40ec-a353-5add1a20d459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:58.028379 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1433/2824] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:58.032553 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398fc50-96ad-4164-b8ae-fe4328839d82 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:58.043119 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398fc50-96ad-4164-b8ae-fe4328839d82 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:58.047679 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a0e23-9815-4a68-8d28-6064848e0969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:58.048271 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a0e23-9815-4a68-8d28-6064848e0969 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.048551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a0e23-9815-4a68-8d28-6064848e0969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.048821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a0e23-9815-4a68-8d28-6064848e0969 admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.049466 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a0e23-9815-4a68-8d28-6064848e0969 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:51:58.049770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a0e23-9815-4a68-8d28-6064848e0969 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:58.051365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a0e23-9815-4a68-8d28-6064848e0969 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:58.062755 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1392/2825] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:51:58.066930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7671f8-4819-40ec-a353-5add1a20d459 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:58.076146 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7671f8-4819-40ec-a353-5add1a20d459 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:58.080742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604c24b-08a8-4a08-8704-306ab2f7342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:58.081328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c24b-08a8-4a08-8704-306ab2f7342c admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.081601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c24b-08a8-4a08-8704-306ab2f7342c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.081870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604c24b-08a8-4a08-8704-306ab2f7342c admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.082522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604c24b-08a8-4a08-8704-306ab2f7342c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:58.084397 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0604c24b-08a8-4a08-8704-306ab2f7342c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:58.179904 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1434/2826] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:58 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:58.183979 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a0e23-9815-4a68-8d28-6064848e0969 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:58.194102 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a0e23-9815-4a68-8d28-6064848e0969 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:58.250818 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8218a39-8b61-4f3b-a06b-5e6b40154a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:58.251353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8218a39-8b61-4f3b-a06b-5e6b40154a3b admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.251911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8218a39-8b61-4f3b-a06b-5e6b40154a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.252185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8218a39-8b61-4f3b-a06b-5e6b40154a3b admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.252805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8218a39-8b61-4f3b-a06b-5e6b40154a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:58.254798 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8218a39-8b61-4f3b-a06b-5e6b40154a3b admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:58.356062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1393/2827] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:58 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:58.360166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604c24b-08a8-4a08-8704-306ab2f7342c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:58.370390 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604c24b-08a8-4a08-8704-306ab2f7342c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:58.374981 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5946fab-972f-49e1-b36d-610bedd2c0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:58.375598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5946fab-972f-49e1-b36d-610bedd2c0fa admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.375874 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5946fab-972f-49e1-b36d-610bedd2c0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.376142 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5946fab-972f-49e1-b36d-610bedd2c0fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.376772 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5946fab-972f-49e1-b36d-610bedd2c0fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:58.378444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5946fab-972f-49e1-b36d-610bedd2c0fa admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:58.388703 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1435/2828] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:58.392562 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8218a39-8b61-4f3b-a06b-5e6b40154a3b admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:58.403040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8218a39-8b61-4f3b-a06b-5e6b40154a3b admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:58.407704 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff21c19-96a1-44ad-b934-106ec48c1e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:58.408297 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff21c19-96a1-44ad-b934-106ec48c1e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.408573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff21c19-96a1-44ad-b934-106ec48c1e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.408843 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff21c19-96a1-44ad-b934-106ec48c1e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.409473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff21c19-96a1-44ad-b934-106ec48c1e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:58.411065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff21c19-96a1-44ad-b934-106ec48c1e22 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:58.421814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1394/2829] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:51:58.432174 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5946fab-972f-49e1-b36d-610bedd2c0fa admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:58.442267 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5946fab-972f-49e1-b36d-610bedd2c0fa admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:58.447729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b148bf-10c0-44fb-9f20-914ce00c38cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:58.448367 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b148bf-10c0-44fb-9f20-914ce00c38cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.448642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b148bf-10c0-44fb-9f20-914ce00c38cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.448909 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b148bf-10c0-44fb-9f20-914ce00c38cd admin admin] PATH_INFO: `/v3/projects/52244ec276074736b79d0c8b2aecc859/users/5bdd21364f1441c084d82aba1ef4f754/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.463730 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b148bf-10c0-44fb-9f20-914ce00c38cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=52244ec276074736b79d0c8b2aecc859, user_id=5bdd21364f1441c084d82aba1ef4f754, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:58.466384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b148bf-10c0-44fb-9f20-914ce00c38cd admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:58.544617 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1436/2830] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:58 2022] PUT /identity/v3/projects/52244ec276074736b79d0c8b2aecc859/users/5bdd21364f1441c084d82aba1ef4f754/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:58.554878 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca0ce-64ab-420e-b5ce-dd77198ba019 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.555561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca0ce-64ab-420e-b5ce-dd77198ba019 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.555561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca0ce-64ab-420e-b5ce-dd77198ba019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.583365 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d2dca0ce-64ab-420e-b5ce-dd77198ba019 None None] MFA Rules not processed for user `5bdd21364f1441c084d82aba1ef4f754`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:58.584565 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d2dca0ce-64ab-420e-b5ce-dd77198ba019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:58.648053 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1395/2831] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:51:58.652591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b148bf-10c0-44fb-9f20-914ce00c38cd admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:58.673737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:58.714103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b148bf-10c0-44fb-9f20-914ce00c38cd admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:58.718726 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28036bc2-13d3-46f5-b20c-d47aa7cabff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:58.719321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28036bc2-13d3-46f5-b20c-d47aa7cabff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.719595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28036bc2-13d3-46f5-b20c-d47aa7cabff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.719862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28036bc2-13d3-46f5-b20c-d47aa7cabff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.720534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28036bc2-13d3-46f5-b20c-d47aa7cabff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:58.722544 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28036bc2-13d3-46f5-b20c-d47aa7cabff2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:58.811686 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1437/2832] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:58 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:58.812835 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dca0ce-64ab-420e-b5ce-dd77198ba019 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:58.824342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dca0ce-64ab-420e-b5ce-dd77198ba019 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:58.836393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:58.880077 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750eefc-88bb-4145-8534-547db64cce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:58.880926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750eefc-88bb-4145-8534-547db64cce7a admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.881394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750eefc-88bb-4145-8534-547db64cce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.881688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750eefc-88bb-4145-8534-547db64cce7a admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.882965 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750eefc-88bb-4145-8534-547db64cce7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:58.885600 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750eefc-88bb-4145-8534-547db64cce7a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:58.944351 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1396/2833] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:58 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:51:58.949008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28036bc2-13d3-46f5-b20c-d47aa7cabff2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:58.959252 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28036bc2-13d3-46f5-b20c-d47aa7cabff2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:58.963896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba93fd-56aa-44c7-af74-2c9073792076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:58.964489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba93fd-56aa-44c7-af74-2c9073792076 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.964765 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba93fd-56aa-44c7-af74-2c9073792076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.965040 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba93fd-56aa-44c7-af74-2c9073792076 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.965678 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba93fd-56aa-44c7-af74-2c9073792076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:58.967310 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba93fd-56aa-44c7-af74-2c9073792076 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:58.978183 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1438/2834] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:58.982121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750eefc-88bb-4145-8534-547db64cce7a admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:58.992191 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750eefc-88bb-4145-8534-547db64cce7a admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:58.997330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30536a94-137e-44d8-8647-e60a36eec021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:58.997924 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30536a94-137e-44d8-8647-e60a36eec021 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:58.998230 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30536a94-137e-44d8-8647-e60a36eec021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:58.998503 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30536a94-137e-44d8-8647-e60a36eec021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:58.999131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30536a94-137e-44d8-8647-e60a36eec021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:59.000688 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30536a94-137e-44d8-8647-e60a36eec021 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:59.010362 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1397/2835] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:59.014352 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba93fd-56aa-44c7-af74-2c9073792076 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:59.024079 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba93fd-56aa-44c7-af74-2c9073792076 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:59.028612 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727ba1a-18ad-4b73-b964-48b430523c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:59.029249 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727ba1a-18ad-4b73-b964-48b430523c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:59.029529 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727ba1a-18ad-4b73-b964-48b430523c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:59.029803 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727ba1a-18ad-4b73-b964-48b430523c7a admin admin] PATH_INFO: `/v3/projects/b43569c5fcb74e56ab484e2394c414b5/users/84a37b38ec9641afba8dc2214befd7bf/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:59.044599 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727ba1a-18ad-4b73-b964-48b430523c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43569c5fcb74e56ab484e2394c414b5, user_id=84a37b38ec9641afba8dc2214befd7bf, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:59.047280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727ba1a-18ad-4b73-b964-48b430523c7a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:59.115357 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1439/2836] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:59 2022] PUT /identity/v3/projects/b43569c5fcb74e56ab484e2394c414b5/users/84a37b38ec9641afba8dc2214befd7bf/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:59.186260 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8eabb-71b0-42a6-baab-f4016707864c None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:59.186549 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8eabb-71b0-42a6-baab-f4016707864c None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:59.186821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8eabb-71b0-42a6-baab-f4016707864c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:59.215262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-07e8eabb-71b0-42a6-baab-f4016707864c None None] MFA Rules not processed for user `84a37b38ec9641afba8dc2214befd7bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:59.216467 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-07e8eabb-71b0-42a6-baab-f4016707864c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:59.278539 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1398/2837] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:59.282934 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727ba1a-18ad-4b73-b964-48b430523c7a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:59.343492 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727ba1a-18ad-4b73-b964-48b430523c7a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:59.348051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840cd11-8125-4607-b77c-5b397cb63e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:59.348642 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840cd11-8125-4607-b77c-5b397cb63e0f admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:59.348915 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840cd11-8125-4607-b77c-5b397cb63e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:59.349181 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840cd11-8125-4607-b77c-5b397cb63e0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:59.350122 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840cd11-8125-4607-b77c-5b397cb63e0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:59.352033 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840cd11-8125-4607-b77c-5b397cb63e0f admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:59.431048 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1440/2838] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:51:59 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:51:59.439009 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8eabb-71b0-42a6-baab-f4016707864c None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:59.450225 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8eabb-71b0-42a6-baab-f4016707864c None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:59.506569 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a1f50-4b15-46d1-9bfa-0b9e7a06c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:59.507173 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a1f50-4b15-46d1-9bfa-0b9e7a06c3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:59.507456 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a1f50-4b15-46d1-9bfa-0b9e7a06c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:59.507732 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a1f50-4b15-46d1-9bfa-0b9e7a06c3ad admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:59.508393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a1f50-4b15-46d1-9bfa-0b9e7a06c3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:59.510323 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a1f50-4b15-46d1-9bfa-0b9e7a06c3ad admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:59.595882 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1399/2839] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:51:59 2022] POST /identity/v3/users => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:51:59.599823 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840cd11-8125-4607-b77c-5b397cb63e0f admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:59.610693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840cd11-8125-4607-b77c-5b397cb63e0f admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:59.615279 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a315832-781a-4587-b5b7-78b5a4d4a515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:59.615863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a315832-781a-4587-b5b7-78b5a4d4a515 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:59.616140 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a315832-781a-4587-b5b7-78b5a4d4a515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:59.616410 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a315832-781a-4587-b5b7-78b5a4d4a515 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:59.617056 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a315832-781a-4587-b5b7-78b5a4d4a515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:59.618704 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a315832-781a-4587-b5b7-78b5a4d4a515 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:59.628868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1441/2840] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:59.632766 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a1f50-4b15-46d1-9bfa-0b9e7a06c3ad admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:59.642440 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a1f50-4b15-46d1-9bfa-0b9e7a06c3ad admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:59.646997 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5608ba-6b8e-406d-ab49-34b40afd2177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:59.647629 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5608ba-6b8e-406d-ab49-34b40afd2177 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:59.647906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5608ba-6b8e-406d-ab49-34b40afd2177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:59.648175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5608ba-6b8e-406d-ab49-34b40afd2177 admin admin] PATH_INFO: `/v3/projects/62610fd412554f728941126c810a0ab6/users/2fb8d886cbb646fd8a8f4cafcc2169cb/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:59.661308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:59.664146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5608ba-6b8e-406d-ab49-34b40afd2177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62610fd412554f728941126c810a0ab6, user_id=2fb8d886cbb646fd8a8f4cafcc2169cb, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:59.667461 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5608ba-6b8e-406d-ab49-34b40afd2177 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:59.720739 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1400/2841] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:59 2022] PUT /identity/v3/projects/62610fd412554f728941126c810a0ab6/users/2fb8d886cbb646fd8a8f4cafcc2169cb/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:59.724710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a315832-781a-4587-b5b7-78b5a4d4a515 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:59.735141 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a315832-781a-4587-b5b7-78b5a4d4a515 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:59.776132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:51:59.790629 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da8797-287d-46ba-b63c-038a0b85687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:59.798578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da8797-287d-46ba-b63c-038a0b85687b admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:59.798578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da8797-287d-46ba-b63c-038a0b85687b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:59.798578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da8797-287d-46ba-b63c-038a0b85687b admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:59.798578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da8797-287d-46ba-b63c-038a0b85687b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:59.798578 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da8797-287d-46ba-b63c-038a0b85687b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:59.803443 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1442/2842] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:51:59.807551 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5608ba-6b8e-406d-ab49-34b40afd2177 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:59.818058 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5608ba-6b8e-406d-ab49-34b40afd2177 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:51:59.826756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07823f1-bfa4-40e3-9094-44c41fe67136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:51:59.826756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07823f1-bfa4-40e3-9094-44c41fe67136 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:59.826756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07823f1-bfa4-40e3-9094-44c41fe67136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:59.826756 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07823f1-bfa4-40e3-9094-44c41fe67136 admin admin] PATH_INFO: `/v3/projects/62610fd412554f728941126c810a0ab6/users/2fb8d886cbb646fd8a8f4cafcc2169cb/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:59.827176 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07823f1-bfa4-40e3-9094-44c41fe67136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62610fd412554f728941126c810a0ab6, user_id=2fb8d886cbb646fd8a8f4cafcc2169cb, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:51:59.829339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07823f1-bfa4-40e3-9094-44c41fe67136 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:51:59.840255 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1401/2843] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:51:59 2022] PUT /identity/v3/projects/62610fd412554f728941126c810a0ab6/users/2fb8d886cbb646fd8a8f4cafcc2169cb/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:51:59.850402 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d235692-4771-4d55-a353-caf0fb5b001c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:51:59.850683 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d235692-4771-4d55-a353-caf0fb5b001c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:51:59.850968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d235692-4771-4d55-a353-caf0fb5b001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:51:59.878478 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2d235692-4771-4d55-a353-caf0fb5b001c None None] MFA Rules not processed for user `2fb8d886cbb646fd8a8f4cafcc2169cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:51:59.879695 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2d235692-4771-4d55-a353-caf0fb5b001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:51:59.949294 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1443/2844] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:51:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:51:59.960509 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07823f1-bfa4-40e3-9094-44c41fe67136 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:51:59.963092 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-e07823f1-bfa4-40e3-9094-44c41fe67136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:00.019294 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07823f1-bfa4-40e3-9094-44c41fe67136 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:00.022977 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95560808-92ad-47d1-808f-8d84336e6f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:00.023585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560808-92ad-47d1-808f-8d84336e6f72 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:00.023854 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560808-92ad-47d1-808f-8d84336e6f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:00.024693 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560808-92ad-47d1-808f-8d84336e6f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:00.079185 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95560808-92ad-47d1-808f-8d84336e6f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:00.080592 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95560808-92ad-47d1-808f-8d84336e6f72 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:00.089180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1402/2845] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:51:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:02.036744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d235692-4771-4d55-a353-caf0fb5b001c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:02.046489 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d235692-4771-4d55-a353-caf0fb5b001c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:02.052628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59f625-7548-4e68-8afe-8a96bc744627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:02.052628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59f625-7548-4e68-8afe-8a96bc744627 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:02.052628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59f625-7548-4e68-8afe-8a96bc744627 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:02.052628 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59f625-7548-4e68-8afe-8a96bc744627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:02.108003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59f625-7548-4e68-8afe-8a96bc744627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:02.109548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59f625-7548-4e68-8afe-8a96bc744627 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:02.118488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1444/2846] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:02.471821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb14c6-2161-4ac8-ad82-fc4304a07def None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:02.472222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb14c6-2161-4ac8-ad82-fc4304a07def None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:02.472354 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb14c6-2161-4ac8-ad82-fc4304a07def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:02.504118 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-9bbb14c6-2161-4ac8-ad82-fc4304a07def None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:02.505317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9bbb14c6-2161-4ac8-ad82-fc4304a07def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:02.561218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1403/2847] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 05:52:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:02.725982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59f625-7548-4e68-8afe-8a96bc744627 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:02.727978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fa59f625-7548-4e68-8afe-8a96bc744627 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:02.802274 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59f625-7548-4e68-8afe-8a96bc744627 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:02.807930 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36cd54-83e4-4528-95f7-22b5ef75746b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:02.810554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36cd54-83e4-4528-95f7-22b5ef75746b service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:02.810554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36cd54-83e4-4528-95f7-22b5ef75746b service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:02.810554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36cd54-83e4-4528-95f7-22b5ef75746b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:02.814072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=100 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:02.817580 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36cd54-83e4-4528-95f7-22b5ef75746b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:02.819447 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36cd54-83e4-4528-95f7-22b5ef75746b service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:02.827230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1445/2848] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:52:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:52:03.558171 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb14c6-2161-4ac8-ad82-fc4304a07def None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:03.559963 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9bbb14c6-2161-4ac8-ad82-fc4304a07def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:03.591791 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb14c6-2161-4ac8-ad82-fc4304a07def None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:03.602475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3bb00-6a3d-4c6c-8a30-9ad68f1cd765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:03.603433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3bb00-6a3d-4c6c-8a30-9ad68f1cd765 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:03.603966 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3bb00-6a3d-4c6c-8a30-9ad68f1cd765 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:03.604518 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3bb00-6a3d-4c6c-8a30-9ad68f1cd765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:03.615330 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3bb00-6a3d-4c6c-8a30-9ad68f1cd765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:03.620432 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3bb00-6a3d-4c6c-8a30-9ad68f1cd765 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:03.629816 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1404/2849] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:08.372425 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36cd54-83e4-4528-95f7-22b5ef75746b service swift] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:08.446321 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36cd54-83e4-4528-95f7-22b5ef75746b service swift] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:08.451361 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:08.452007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:08.452289 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:08.452565 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] PATH_INFO: `/v3/users/5bdd21364f1441c084d82aba1ef4f754` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:08.453710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bdd21364f1441c084d82aba1ef4f754)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:08.455854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:08.689846 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bdd21364f1441c084d82aba1ef4f754'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:08.706127 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bdd21364f1441c084d82aba1ef4f754'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:08.707132 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bdd21364f1441c084d82aba1ef4f754'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:08.781716 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bdd21364f1441c084d82aba1ef4f754'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:08.783870 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bdd21364f1441c084d82aba1ef4f754'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:08.816550 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1446/2850] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:52:08 2022] DELETE /identity/v3/users/5bdd21364f1441c084d82aba1ef4f754 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:08.822544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64eeb-26d6-4f76-a303-321dffe71695 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:08.827121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64eeb-26d6-4f76-a303-321dffe71695 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:08.827121 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64eeb-26d6-4f76-a303-321dffe71695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:08.847241 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-b4f64eeb-26d6-4f76-a303-321dffe71695 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:08.850086 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b4f64eeb-26d6-4f76-a303-321dffe71695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:08.862684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:08.927291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1405/2851] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 05:52:08.934475 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:08.936358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:09.002206 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460fccb-2ee4-4cec-b4fe-43544b1e3246 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:09.006979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa96586-2014-4d67-bb18-a926c9547f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:09.007711 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa96586-2014-4d67-bb18-a926c9547f3d service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:09.008086 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa96586-2014-4d67-bb18-a926c9547f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:09.008460 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa96586-2014-4d67-bb18-a926c9547f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:09.019041 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa96586-2014-4d67-bb18-a926c9547f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:09.020667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa96586-2014-4d67-bb18-a926c9547f3d service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:09.038248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1447/2852] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:09.684033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64eeb-26d6-4f76-a303-321dffe71695 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:09.685615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-b4f64eeb-26d6-4f76-a303-321dffe71695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:09.706051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64eeb-26d6-4f76-a303-321dffe71695 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:09.706051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:09.706051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:09.706051 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:09.707852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] PATH_INFO: `/v3/projects/52244ec276074736b79d0c8b2aecc859` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:09.707852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52244ec276074736b79d0c8b2aecc859)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:09.708088 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:09.839891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa96586-2014-4d67-bb18-a926c9547f3d service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:09.839891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-aaa96586-2014-4d67-bb18-a926c9547f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:09.844111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52244ec276074736b79d0c8b2aecc859', 'request_id': 'req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:09.852363 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52244ec276074736b79d0c8b2aecc859', 'request_id': 'req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:09.855466 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa96586-2014-4d67-bb18-a926c9547f3d service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:09.875099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:09.932671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:09.933725 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:09.934234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:09.934500 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] PATH_INFO: `/v3/users/41860290eb484abd96c9f8e33cecc180` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:09.936399 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41860290eb484abd96c9f8e33cecc180)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:09.939136 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52244ec276074736b79d0c8b2aecc859', 'request_id': 'req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:09.939957 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:09.953923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] The token cache is being invalidate because project 52244ec276074736b79d0c8b2aecc859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:52:09.958489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:09.958489 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:09.965840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1406/2853] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:52:09 2022] DELETE /identity/v3/projects/52244ec276074736b79d0c8b2aecc859 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:09.967369 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:09.968450 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:09.984276 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f7fb9-968f-4274-88fe-25aaef4b5cf6 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:09.989577 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:09.990370 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:09.990721 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:09.991111 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] PATH_INFO: `/v3/users/84a37b38ec9641afba8dc2214befd7bf` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:09.992346 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a37b38ec9641afba8dc2214befd7bf)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:09.994642 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:10.157208 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41860290eb484abd96c9f8e33cecc180'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:10.173844 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41860290eb484abd96c9f8e33cecc180'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:10.175083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41860290eb484abd96c9f8e33cecc180'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:10.214893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a37b38ec9641afba8dc2214befd7bf'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:10.215658 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a37b38ec9641afba8dc2214befd7bf'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:10.230910 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a37b38ec9641afba8dc2214befd7bf'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:10.254050 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41860290eb484abd96c9f8e33cecc180'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:10.254804 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-77a50fa9-1bf5-4aba-93ba-efc3aa990355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41860290eb484abd96c9f8e33cecc180'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:10.270737 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1448/2854] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:52:09 2022] DELETE /identity/v3/users/41860290eb484abd96c9f8e33cecc180 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:52:10.275913 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5240b-eab0-41bd-91b0-747b50086ba6 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:10.276881 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5240b-eab0-41bd-91b0-747b50086ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:10.277398 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5240b-eab0-41bd-91b0-747b50086ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:10.300242 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-52a5240b-eab0-41bd-91b0-747b50086ba6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:10.301817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-52a5240b-eab0-41bd-91b0-747b50086ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:10.304707 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a37b38ec9641afba8dc2214befd7bf'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:10.305564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a37b38ec9641afba8dc2214befd7bf'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:10.329499 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1407/2855] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:52:09 2022] DELETE /identity/v3/users/84a37b38ec9641afba8dc2214befd7bf => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:10.399868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1449/2856] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:10.400603 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:10.402094 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:10.426923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:10.483533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbf6e4-90c4-4ba4-80b6-8554f5e513d2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:10.487388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98036497-6df4-466d-86a3-561b69edb435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:10.488047 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98036497-6df4-466d-86a3-561b69edb435 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:10.488331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98036497-6df4-466d-86a3-561b69edb435 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:10.488604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98036497-6df4-466d-86a3-561b69edb435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:10.563626 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98036497-6df4-466d-86a3-561b69edb435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:10.565318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98036497-6df4-466d-86a3-561b69edb435 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:10.579641 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1408/2857] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:10.956439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5240b-eab0-41bd-91b0-747b50086ba6 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:10.963094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-52a5240b-eab0-41bd-91b0-747b50086ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:10.985204 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5240b-eab0-41bd-91b0-747b50086ba6 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:11.008001 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:11.009461 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:11.009797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:11.010082 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] PATH_INFO: `/v3/projects/b43569c5fcb74e56ab484e2394c414b5` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:11.011339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43569c5fcb74e56ab484e2394c414b5)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:11.013919 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:11.117857 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:11.138894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43569c5fcb74e56ab484e2394c414b5', 'request_id': 'req-f32191c4-4515-43ce-a488-4c5d834d90b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.145577 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43569c5fcb74e56ab484e2394c414b5', 'request_id': 'req-f32191c4-4515-43ce-a488-4c5d834d90b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.164747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98036497-6df4-466d-86a3-561b69edb435 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:11.166339 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-98036497-6df4-466d-86a3-561b69edb435 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:11.185151 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98036497-6df4-466d-86a3-561b69edb435 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:11.196867 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43569c5fcb74e56ab484e2394c414b5', 'request_id': 'req-f32191c4-4515-43ce-a488-4c5d834d90b1'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.216117 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] The token cache is being invalidate because project b43569c5fcb74e56ab484e2394c414b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:52:11.216419 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.217094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.223845 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1450/2858] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:52:10 2022] DELETE /identity/v3/projects/b43569c5fcb74e56ab484e2394c414b5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:11.226169 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:11.228452 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:11.255604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:11.256495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:11.256789 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:11.257163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] PATH_INFO: `/v3/projects/fdcd39c77f8d4378bdc8d6149ee3779b` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:11.258532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdcd39c77f8d4378bdc8d6149ee3779b)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:11.261298 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:11.329534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32191c4-4515-43ce-a488-4c5d834d90b1 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:11.337287 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:11.338192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:11.338525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:11.338884 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] PATH_INFO: `/v3/users/2fb8d886cbb646fd8a8f4cafcc2169cb` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:11.340282 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb8d886cbb646fd8a8f4cafcc2169cb)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:11.342413 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:11.362394 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdcd39c77f8d4378bdc8d6149ee3779b', 'request_id': 'req-e07eb98f-01a0-47fb-8290-e72a55efdc1a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.369705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdcd39c77f8d4378bdc8d6149ee3779b', 'request_id': 'req-e07eb98f-01a0-47fb-8290-e72a55efdc1a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.412949 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdcd39c77f8d4378bdc8d6149ee3779b', 'request_id': 'req-e07eb98f-01a0-47fb-8290-e72a55efdc1a'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.415758 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:11.425829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] The token cache is being invalidate because project fdcd39c77f8d4378bdc8d6149ee3779b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:52:11.426126 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.426909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-e07eb98f-01a0-47fb-8290-e72a55efdc1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.428633 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1409/2859] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:52:11 2022] DELETE /identity/v3/projects/fdcd39c77f8d4378bdc8d6149ee3779b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:11.434652 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eca379-0903-48de-ad4b-e75b209fdee6 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:11.434905 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eca379-0903-48de-ad4b-e75b209fdee6 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:11.435180 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eca379-0903-48de-ad4b-e75b209fdee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:11.456372 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-05eca379-0903-48de-ad4b-e75b209fdee6 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:11.457506 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-05eca379-0903-48de-ad4b-e75b209fdee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:11.523426 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb8d886cbb646fd8a8f4cafcc2169cb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.524912 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1410/2860] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:11.533221 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe762-1dea-433f-b222-c539b5171bb9 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:11.533561 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe762-1dea-433f-b222-c539b5171bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:11.537101 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fe762-1dea-433f-b222-c539b5171bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:11.539885 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb8d886cbb646fd8a8f4cafcc2169cb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.540910 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb8d886cbb646fd8a8f4cafcc2169cb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.564573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-ca0fe762-1dea-433f-b222-c539b5171bb9 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:11.565802 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-ca0fe762-1dea-433f-b222-c539b5171bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:11.575380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1411/2861] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:11.583393 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fe762-1dea-433f-b222-c539b5171bb9 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:11.596422 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fe762-1dea-433f-b222-c539b5171bb9 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:11.602839 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f28034-bd78-49c9-98e3-c8de486b34da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:11.603763 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f28034-bd78-49c9-98e3-c8de486b34da admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:11.604195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f28034-bd78-49c9-98e3-c8de486b34da admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:11.604596 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f28034-bd78-49c9-98e3-c8de486b34da admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:11.605571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f28034-bd78-49c9-98e3-c8de486b34da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:52:11.605947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f28034-bd78-49c9-98e3-c8de486b34da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:11.607674 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f28034-bd78-49c9-98e3-c8de486b34da admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:11.628976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1412/2862] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:52:11.632770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb8d886cbb646fd8a8f4cafcc2169cb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.634554 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb8d886cbb646fd8a8f4cafcc2169cb'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:11.635765 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f28034-bd78-49c9-98e3-c8de486b34da admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:11.647420 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f28034-bd78-49c9-98e3-c8de486b34da admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:11.650091 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1451/2863] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:52:11 2022] DELETE /identity/v3/users/2fb8d886cbb646fd8a8f4cafcc2169cb => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:11.655605 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27751b9b-3507-4e8a-b107-0920968a5036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:11.656417 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27751b9b-3507-4e8a-b107-0920968a5036 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:11.656982 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27751b9b-3507-4e8a-b107-0920968a5036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:11.657435 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27751b9b-3507-4e8a-b107-0920968a5036 admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:11.658252 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27751b9b-3507-4e8a-b107-0920968a5036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:11.660532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27751b9b-3507-4e8a-b107-0920968a5036 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:11.746573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1413/2864] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:52:11 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:52:11.751243 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:11.813049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:11.835157 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ca0be-ea60-4915-bbf8-e4672b6b5137 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:11.922323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15399bf8-0654-4b0a-a6a8-9f52e8b5c181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:11.923667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15399bf8-0654-4b0a-a6a8-9f52e8b5c181 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:11.923667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15399bf8-0654-4b0a-a6a8-9f52e8b5c181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:11.923667 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15399bf8-0654-4b0a-a6a8-9f52e8b5c181 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:11.924156 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15399bf8-0654-4b0a-a6a8-9f52e8b5c181 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:11.926199 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15399bf8-0654-4b0a-a6a8-9f52e8b5c181 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:12.014055 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1452/2865] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:52:11 2022] POST /identity/v3/users => generated 288 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:52:12.017630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27751b9b-3507-4e8a-b107-0920968a5036 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:12.029656 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27751b9b-3507-4e8a-b107-0920968a5036 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:12.035389 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdc576-a8a0-4ec5-a6cf-83b90b9f3d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:12.035815 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdc576-a8a0-4ec5-a6cf-83b90b9f3d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.036195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdc576-a8a0-4ec5-a6cf-83b90b9f3d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.036768 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdc576-a8a0-4ec5-a6cf-83b90b9f3d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.037552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdc576-a8a0-4ec5-a6cf-83b90b9f3d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:12.039666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdc576-a8a0-4ec5-a6cf-83b90b9f3d45 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:12.052146 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1414/2866] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:52:12.057198 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15399bf8-0654-4b0a-a6a8-9f52e8b5c181 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:12.069938 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15399bf8-0654-4b0a-a6a8-9f52e8b5c181 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:12.077939 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056dc41-0171-44e9-83da-a7d2cd609e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:12.078680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056dc41-0171-44e9-83da-a7d2cd609e9c admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.079022 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056dc41-0171-44e9-83da-a7d2cd609e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.079380 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056dc41-0171-44e9-83da-a7d2cd609e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.080139 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056dc41-0171-44e9-83da-a7d2cd609e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:12.081892 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056dc41-0171-44e9-83da-a7d2cd609e9c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:12.093541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1453/2867] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:52:12.098342 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdc576-a8a0-4ec5-a6cf-83b90b9f3d45 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:12.114662 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdc576-a8a0-4ec5-a6cf-83b90b9f3d45 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:12.119363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056dc41-0171-44e9-83da-a7d2cd609e9c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:12.122164 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3056dc41-0171-44e9-83da-a7d2cd609e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:12.128353 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869b44b-c830-4112-842b-8416538e911d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:12.133097 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869b44b-c830-4112-842b-8416538e911d admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.133573 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869b44b-c830-4112-842b-8416538e911d admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.133805 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869b44b-c830-4112-842b-8416538e911d admin admin] PATH_INFO: `/v3/projects/ae91802cb7db42e387a53bb5a38b2c43/users/c30fb5d775714ca5a3713af17a4e61a0/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.152175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056dc41-0171-44e9-83da-a7d2cd609e9c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:12.156161 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869b44b-c830-4112-842b-8416538e911d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae91802cb7db42e387a53bb5a38b2c43, user_id=c30fb5d775714ca5a3713af17a4e61a0, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:12.159163 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869b44b-c830-4112-842b-8416538e911d admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:12.160311 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:12.161030 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.161941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.162258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] PATH_INFO: `/v3/projects/62610fd412554f728941126c810a0ab6` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.167600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62610fd412554f728941126c810a0ab6)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:12.167600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:12.240288 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1415/2868] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:52:12 2022] PUT /identity/v3/projects/ae91802cb7db42e387a53bb5a38b2c43/users/c30fb5d775714ca5a3713af17a4e61a0/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:12.242615 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efedaf-3619-439f-911a-2593cddaf094 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.246510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efedaf-3619-439f-911a-2593cddaf094 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.246510 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81efedaf-3619-439f-911a-2593cddaf094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.283175 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62610fd412554f728941126c810a0ab6', 'request_id': 'req-eb6ca37a-2881-4937-b068-ab76708e6730'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:12.283812 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-81efedaf-3619-439f-911a-2593cddaf094 None None] MFA Rules not processed for user `c30fb5d775714ca5a3713af17a4e61a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:12.285132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-81efedaf-3619-439f-911a-2593cddaf094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:12.294246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62610fd412554f728941126c810a0ab6', 'request_id': 'req-eb6ca37a-2881-4937-b068-ab76708e6730'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:12.350384 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62610fd412554f728941126c810a0ab6', 'request_id': 'req-eb6ca37a-2881-4937-b068-ab76708e6730'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:12.364482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1416/2869] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:12.377148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] The token cache is being invalidate because project 62610fd412554f728941126c810a0ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:52:12.380375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:12.380504 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efedaf-3619-439f-911a-2593cddaf094 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:12.380618 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-eb6ca37a-2881-4937-b068-ab76708e6730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:12.380714 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-81efedaf-3619-439f-911a-2593cddaf094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:12.380798 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1454/2870] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:52:12 2022] DELETE /identity/v3/projects/62610fd412554f728941126c810a0ab6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:52:12.385326 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc298-43f2-452c-813c-3ef4482692ec None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.385707 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc298-43f2-452c-813c-3ef4482692ec None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.386062 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc298-43f2-452c-813c-3ef4482692ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.391275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:12.411671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-c3bdc298-43f2-452c-813c-3ef4482692ec None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:12.413371 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c3bdc298-43f2-452c-813c-3ef4482692ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:12.464202 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81efedaf-3619-439f-911a-2593cddaf094 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:12.468120 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c3c18-5c6e-4c04-bdae-0b7439a2b869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:12.468750 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c3c18-5c6e-4c04-bdae-0b7439a2b869 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.469025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c3c18-5c6e-4c04-bdae-0b7439a2b869 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.469296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c3c18-5c6e-4c04-bdae-0b7439a2b869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.471237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f86c3c18-5c6e-4c04-bdae-0b7439a2b869 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:12.485644 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c3c18-5c6e-4c04-bdae-0b7439a2b869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:12.487312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c3c18-5c6e-4c04-bdae-0b7439a2b869 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:12.495763 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:12.513019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1417/2871] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:12.523830 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1455/2872] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:12.549734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879acc-a451-4341-86c0-2bab7f23a4a2 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.549734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879acc-a451-4341-86c0-2bab7f23a4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.549734 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65879acc-a451-4341-86c0-2bab7f23a4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.579412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-65879acc-a451-4341-86c0-2bab7f23a4a2 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:12.581140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-65879acc-a451-4341-86c0-2bab7f23a4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:12.596061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdc298-43f2-452c-813c-3ef4482692ec None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:12.598315 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1418/2873] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:12.606878 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdc298-43f2-452c-813c-3ef4482692ec None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:12.612258 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7f337-2aaf-4df6-b464-297f7307f8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:12.613068 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7f337-2aaf-4df6-b464-297f7307f8bf admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.613445 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7f337-2aaf-4df6-b464-297f7307f8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.613817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7f337-2aaf-4df6-b464-297f7307f8bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.614601 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7f337-2aaf-4df6-b464-297f7307f8bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:52:12.615094 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7f337-2aaf-4df6-b464-297f7307f8bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:12.617147 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7f337-2aaf-4df6-b464-297f7307f8bf admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:12.632979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1456/2874] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:52:12.638038 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65879acc-a451-4341-86c0-2bab7f23a4a2 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:12.648065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65879acc-a451-4341-86c0-2bab7f23a4a2 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:12.654281 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae075e16-63e4-440e-9ec1-5d690119a6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:12.654923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae075e16-63e4-440e-9ec1-5d690119a6ce admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.655270 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae075e16-63e4-440e-9ec1-5d690119a6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.655602 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae075e16-63e4-440e-9ec1-5d690119a6ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.657011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae075e16-63e4-440e-9ec1-5d690119a6ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:12.659366 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae075e16-63e4-440e-9ec1-5d690119a6ce admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:12.873025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1419/2875] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:52:12 2022] POST /identity/v3/projects => generated 373 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 05:52:12.877008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7f337-2aaf-4df6-b464-297f7307f8bf admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:12.888328 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7f337-2aaf-4df6-b464-297f7307f8bf admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:12.947998 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321fcd-945d-4c28-9309-ee4b7071b0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:12.948595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321fcd-945d-4c28-9309-ee4b7071b0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:12.948914 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321fcd-945d-4c28-9309-ee4b7071b0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:12.949189 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321fcd-945d-4c28-9309-ee4b7071b0e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:12.949817 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321fcd-945d-4c28-9309-ee4b7071b0e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:12.951764 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321fcd-945d-4c28-9309-ee4b7071b0e5 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:13.082548 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1457/2876] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:52:12 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:52:13.086921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae075e16-63e4-440e-9ec1-5d690119a6ce admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:13.097968 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae075e16-63e4-440e-9ec1-5d690119a6ce admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:13.103482 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297b572-9ed0-4b1f-9b63-781f361e3abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:13.108414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297b572-9ed0-4b1f-9b63-781f361e3abe admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:13.108414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297b572-9ed0-4b1f-9b63-781f361e3abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:13.108414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297b572-9ed0-4b1f-9b63-781f361e3abe admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:13.108414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297b572-9ed0-4b1f-9b63-781f361e3abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:13.108414 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297b572-9ed0-4b1f-9b63-781f361e3abe admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:13.117711 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1420/2877] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:52:13.122388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321fcd-945d-4c28-9309-ee4b7071b0e5 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:13.133008 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321fcd-945d-4c28-9309-ee4b7071b0e5 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:13.139591 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268926a6-d62b-4e20-904c-8283df4e8e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:13.140626 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268926a6-d62b-4e20-904c-8283df4e8e51 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:13.140944 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268926a6-d62b-4e20-904c-8283df4e8e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:13.141329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268926a6-d62b-4e20-904c-8283df4e8e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:13.141929 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268926a6-d62b-4e20-904c-8283df4e8e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:13.143614 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268926a6-d62b-4e20-904c-8283df4e8e51 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:13.159571 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297b572-9ed0-4b1f-9b63-781f361e3abe admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:13.160553 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1458/2878] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:52:13.170929 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297b572-9ed0-4b1f-9b63-781f361e3abe admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:13.177418 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566a1d1-9f48-44b8-9b60-c1b76168a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:13.178268 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566a1d1-9f48-44b8-9b60-c1b76168a395 admin admin] REQUEST_METHOD: `PUT` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:13.178457 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566a1d1-9f48-44b8-9b60-c1b76168a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:13.178761 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566a1d1-9f48-44b8-9b60-c1b76168a395 admin admin] PATH_INFO: `/v3/projects/c225dfb273d64b3e94f7dae792e4cd9d/users/92f4eb565bdf49f197d6b2b4a3f17cc8/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:13.198475 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566a1d1-9f48-44b8-9b60-c1b76168a395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c225dfb273d64b3e94f7dae792e4cd9d, user_id=92f4eb565bdf49f197d6b2b4a3f17cc8, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:13.201906 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566a1d1-9f48-44b8-9b60-c1b76168a395 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:13.298514 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1421/2879] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:52:13 2022] PUT /identity/v3/projects/c225dfb273d64b3e94f7dae792e4cd9d/users/92f4eb565bdf49f197d6b2b4a3f17cc8/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 05:52:13.299735 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de042ef-c64a-4751-b55f-7d4467913643 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:13.300021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de042ef-c64a-4751-b55f-7d4467913643 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:13.300298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de042ef-c64a-4751-b55f-7d4467913643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:13.324301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-7de042ef-c64a-4751-b55f-7d4467913643 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:13.326388 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7de042ef-c64a-4751-b55f-7d4467913643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:13.428534 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1459/2880] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:13.440753 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566a1d1-9f48-44b8-9b60-c1b76168a395 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:13.442651 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-4566a1d1-9f48-44b8-9b60-c1b76168a395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:13.546654 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566a1d1-9f48-44b8-9b60-c1b76168a395 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:13.551556 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac901d4-0622-41c3-b3dd-3ffbcb713d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:13.552227 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac901d4-0622-41c3-b3dd-3ffbcb713d3f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:13.552630 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac901d4-0622-41c3-b3dd-3ffbcb713d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:13.553107 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac901d4-0622-41c3-b3dd-3ffbcb713d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:13.564232 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac901d4-0622-41c3-b3dd-3ffbcb713d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:13.567552 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac901d4-0622-41c3-b3dd-3ffbcb713d3f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:13.576781 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1422/2881] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:20.103861 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f534-b83a-48bc-9a42-8cba49acdaf3 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:20.104510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f534-b83a-48bc-9a42-8cba49acdaf3 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:20.104917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115f534-b83a-48bc-9a42-8cba49acdaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:20.156423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-f115f534-b83a-48bc-9a42-8cba49acdaf3 None None] MFA Rules not processed for user `92f4eb565bdf49f197d6b2b4a3f17cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:20.157732 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-f115f534-b83a-48bc-9a42-8cba49acdaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:20.273195 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:20.318894 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1460/2882] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:20.323023 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac901d4-0622-41c3-b3dd-3ffbcb713d3f service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:20.342533 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac901d4-0622-41c3-b3dd-3ffbcb713d3f service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:20.349584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bc3bd-faa0-4369-8d9d-6cfb8792134f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:20.350494 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bc3bd-faa0-4369-8d9d-6cfb8792134f admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:20.351036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bc3bd-faa0-4369-8d9d-6cfb8792134f admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:20.351564 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bc3bd-faa0-4369-8d9d-6cfb8792134f admin admin] PATH_INFO: `/v3/projects` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:20.352893 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bc3bd-faa0-4369-8d9d-6cfb8792134f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:20.355010 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bc3bd-faa0-4369-8d9d-6cfb8792134f admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:20.459260 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1423/2883] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:52:20 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:52:20.460842 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115f534-b83a-48bc-9a42-8cba49acdaf3 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:20.483693 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115f534-b83a-48bc-9a42-8cba49acdaf3 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:20.573467 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd68e8-29a2-4d92-8f68-7ffaad73112b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:20.574386 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd68e8-29a2-4d92-8f68-7ffaad73112b admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:20.575072 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd68e8-29a2-4d92-8f68-7ffaad73112b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:20.575512 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd68e8-29a2-4d92-8f68-7ffaad73112b admin admin] PATH_INFO: `/v3/users` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:20.576263 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd68e8-29a2-4d92-8f68-7ffaad73112b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:20.579828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd68e8-29a2-4d92-8f68-7ffaad73112b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:20.729576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1461/2884] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:52:20 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:52:20.731821 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bc3bd-faa0-4369-8d9d-6cfb8792134f admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:20.751790 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bc3bd-faa0-4369-8d9d-6cfb8792134f admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:20.760891 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce2254-c29f-486e-9a81-eed108fbe1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:20.762794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce2254-c29f-486e-9a81-eed108fbe1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:20.763195 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce2254-c29f-486e-9a81-eed108fbe1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:20.763827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce2254-c29f-486e-9a81-eed108fbe1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:20.765237 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce2254-c29f-486e-9a81-eed108fbe1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:20.769272 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce2254-c29f-486e-9a81-eed108fbe1a5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:20.797129 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd68e8-29a2-4d92-8f68-7ffaad73112b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:20.798262 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1424/2885] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:52:20.820518 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd68e8-29a2-4d92-8f68-7ffaad73112b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:20.830797 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3999500-161a-40a7-bed5-838210c9291a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:20.831576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999500-161a-40a7-bed5-838210c9291a admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:20.831979 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999500-161a-40a7-bed5-838210c9291a admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:20.832575 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3999500-161a-40a7-bed5-838210c9291a admin admin] PATH_INFO: `/v3/projects/a460d3b4dfa14b3ba28c577fa7c30f60/users/237d6b5f9a2e4ed1966caf70e88bcf68/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:20.859246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3999500-161a-40a7-bed5-838210c9291a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a460d3b4dfa14b3ba28c577fa7c30f60, user_id=237d6b5f9a2e4ed1966caf70e88bcf68, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:20.859246 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3999500-161a-40a7-bed5-838210c9291a admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:21.031380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce2254-c29f-486e-9a81-eed108fbe1a5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:21.037841 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1462/2886] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:52:20 2022] PUT /identity/v3/projects/a460d3b4dfa14b3ba28c577fa7c30f60/users/237d6b5f9a2e4ed1966caf70e88bcf68/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:21.045031 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce2254-c29f-486e-9a81-eed108fbe1a5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:21.060012 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:21.169676 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f581aaa-1e36-4c5b-8ab1-ecced409dca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:21.170351 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f581aaa-1e36-4c5b-8ab1-ecced409dca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:21.170638 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f581aaa-1e36-4c5b-8ab1-ecced409dca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:21.170916 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f581aaa-1e36-4c5b-8ab1-ecced409dca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:21.171584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f581aaa-1e36-4c5b-8ab1-ecced409dca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:21.173242 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f581aaa-1e36-4c5b-8ab1-ecced409dca4 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:21.187794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1425/2887] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:52:21.192207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3999500-161a-40a7-bed5-838210c9291a admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:21.207970 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3999500-161a-40a7-bed5-838210c9291a admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:21.213773 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387747b3-4be2-4b3a-8900-9159989b028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:21.214491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387747b3-4be2-4b3a-8900-9159989b028b admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:21.214776 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387747b3-4be2-4b3a-8900-9159989b028b admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:21.215051 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387747b3-4be2-4b3a-8900-9159989b028b admin admin] PATH_INFO: `/v3/projects/a460d3b4dfa14b3ba28c577fa7c30f60/users/237d6b5f9a2e4ed1966caf70e88bcf68/roles/d0a94c98a6c84c18a651e6b8ad809d04` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:21.217254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387747b3-4be2-4b3a-8900-9159989b028b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a460d3b4dfa14b3ba28c577fa7c30f60, user_id=237d6b5f9a2e4ed1966caf70e88bcf68, role_id=d0a94c98a6c84c18a651e6b8ad809d04)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:21.219987 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387747b3-4be2-4b3a-8900-9159989b028b admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:21.232541 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1463/2888] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:52:21 2022] PUT /identity/v3/projects/a460d3b4dfa14b3ba28c577fa7c30f60/users/237d6b5f9a2e4ed1966caf70e88bcf68/roles/d0a94c98a6c84c18a651e6b8ad809d04 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:24.295836 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42d781-9160-4530-9f17-d6fa647e3618 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:24.296710 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42d781-9160-4530-9f17-d6fa647e3618 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:24.297019 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42d781-9160-4530-9f17-d6fa647e3618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:24.318917 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-1d42d781-9160-4530-9f17-d6fa647e3618 None None] MFA Rules not processed for user `5cf157961e3d45d5ad861f9b4f51ab50`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:24.320151 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1d42d781-9160-4530-9f17-d6fa647e3618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:24.387840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1426/2889] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 05:52:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:24.405506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387747b3-4be2-4b3a-8900-9159989b028b admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:24.405506 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-387747b3-4be2-4b3a-8900-9159989b028b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:24.494562 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387747b3-4be2-4b3a-8900-9159989b028b admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:24.502329 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7c63b-bec1-41cf-a358-1ebcdf626fc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '783ad0cd9721409e8a7114cecef16aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:24.505280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c63b-bec1-41cf-a358-1ebcdf626fc5 service swift] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:24.505280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c63b-bec1-41cf-a358-1ebcdf626fc5 service swift] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:24.505280 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c63b-bec1-41cf-a358-1ebcdf626fc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:24.513543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7c63b-bec1-41cf-a358-1ebcdf626fc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:24.515200 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7c63b-bec1-41cf-a358-1ebcdf626fc5 service swift] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:24.529786 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1464/2890] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 05:52:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 05:52:26.077948 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42d781-9160-4530-9f17-d6fa647e3618 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:26.081827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-1d42d781-9160-4530-9f17-d6fa647e3618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:26.244016 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42d781-9160-4530-9f17-d6fa647e3618 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:26.249923 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:26.253309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:26.253309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:26.253309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] PATH_INFO: `/v3/users/c30fb5d775714ca5a3713af17a4e61a0` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:26.253309 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c30fb5d775714ca5a3713af17a4e61a0)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:26.256581 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:26.414296 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c30fb5d775714ca5a3713af17a4e61a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:26.414696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c30fb5d775714ca5a3713af17a4e61a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:26.427352 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:26.433928 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c30fb5d775714ca5a3713af17a4e61a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:26.501897 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c30fb5d775714ca5a3713af17a4e61a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:26.503082 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c30fb5d775714ca5a3713af17a4e61a0'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:26.532378 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1427/2891] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:52:26 2022] DELETE /identity/v3/users/c30fb5d775714ca5a3713af17a4e61a0 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:26.540694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2237e9-e0aa-423c-9d42-41415d95a26a None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:26.540694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2237e9-e0aa-423c-9d42-41415d95a26a None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:26.540694 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2237e9-e0aa-423c-9d42-41415d95a26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:26.564071 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-ed2237e9-e0aa-423c-9d42-41415d95a26a None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:26.565234 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ed2237e9-e0aa-423c-9d42-41415d95a26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:26.602950 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:26.654239 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1465/2892] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:26.663717 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:26.665486 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:26.740588 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096296ca-58e0-42ae-a358-c9a2c9d176f5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:26.745067 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10578531-8436-4659-9fc4-729528abbdf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:26.745697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10578531-8436-4659-9fc4-729528abbdf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:26.745976 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10578531-8436-4659-9fc4-729528abbdf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:26.746314 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10578531-8436-4659-9fc4-729528abbdf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:26.766328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10578531-8436-4659-9fc4-729528abbdf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:26.766328 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10578531-8436-4659-9fc4-729528abbdf5 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:26.769183 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1428/2893] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:27.441362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2237e9-e0aa-423c-9d42-41415d95a26a None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:27.443317 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ed2237e9-e0aa-423c-9d42-41415d95a26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:27.473003 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2237e9-e0aa-423c-9d42-41415d95a26a None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:27.482444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:27.483257 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:27.483656 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:27.484012 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] PATH_INFO: `/v3/projects/ae91802cb7db42e387a53bb5a38b2c43` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:27.485555 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae91802cb7db42e387a53bb5a38b2c43)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:27.487634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:27.635507 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae91802cb7db42e387a53bb5a38b2c43', 'request_id': 'req-fe46e298-8ffa-4815-99b6-d503e7843f6c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:27.660705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae91802cb7db42e387a53bb5a38b2c43', 'request_id': 'req-fe46e298-8ffa-4815-99b6-d503e7843f6c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:27.733487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae91802cb7db42e387a53bb5a38b2c43', 'request_id': 'req-fe46e298-8ffa-4815-99b6-d503e7843f6c'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:27.751543 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] The token cache is being invalidate because project ae91802cb7db42e387a53bb5a38b2c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:52:27.751854 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:27.761491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:27.761491 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1466/2894] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:52:27 2022] DELETE /identity/v3/projects/ae91802cb7db42e387a53bb5a38b2c43 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:27.766036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b7103-3259-47f2-bd1d-b7d8eed90467 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:27.766036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b7103-3259-47f2-bd1d-b7d8eed90467 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:27.766036 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b7103-3259-47f2-bd1d-b7d8eed90467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:27.800169 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-960b7103-3259-47f2-bd1d-b7d8eed90467 None None] MFA Rules not processed for user `237d6b5f9a2e4ed1966caf70e88bcf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:27.803534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-960b7103-3259-47f2-bd1d-b7d8eed90467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:27.945857 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1429/2895] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:27.960710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:27.962988 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:28.081019 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46e298-8ffa-4815-99b6-d503e7843f6c admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:28.089891 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698adea-bbdb-40d3-a324-123aa3e81e9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:28.090519 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698adea-bbdb-40d3-a324-123aa3e81e9f service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:28.090793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698adea-bbdb-40d3-a324-123aa3e81e9f service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:28.091112 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698adea-bbdb-40d3-a324-123aa3e81e9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:28.092917 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d698adea-bbdb-40d3-a324-123aa3e81e9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:28.138391 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:28.157173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698adea-bbdb-40d3-a324-123aa3e81e9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:28.158965 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698adea-bbdb-40d3-a324-123aa3e81e9f service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:28.171302 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1467/2896] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:28.172285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b7103-3259-47f2-bd1d-b7d8eed90467 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:28.178194 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-960b7103-3259-47f2-bd1d-b7d8eed90467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:28.260479 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b7103-3259-47f2-bd1d-b7d8eed90467 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:28.264962 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:28.265582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:28.265859 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:28.266129 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] PATH_INFO: `/v3/users/e04f49a794c5487ba372cf1baaf67402` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:28.267280 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e04f49a794c5487ba372cf1baaf67402)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:28.269231 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:28.271530 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:28.418731 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e04f49a794c5487ba372cf1baaf67402'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:28.419852 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e04f49a794c5487ba372cf1baaf67402'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:28.433609 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e04f49a794c5487ba372cf1baaf67402'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:28.521310 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e04f49a794c5487ba372cf1baaf67402'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:28.521941 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-68052e36-f491-4705-9c20-76c3830f8b72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e04f49a794c5487ba372cf1baaf67402'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:28.543989 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1430/2897] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:52:28 2022] DELETE /identity/v3/users/e04f49a794c5487ba372cf1baaf67402 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:28.809227 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698adea-bbdb-40d3-a324-123aa3e81e9f service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:28.823532 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698adea-bbdb-40d3-a324-123aa3e81e9f service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:28.887576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:28.888231 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:28.888513 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:28.888792 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] PATH_INFO: `/v3/projects/a1efce1ebdf1425396c2babae74fe6ce` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:28.889968 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1efce1ebdf1425396c2babae74fe6ce)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:28.892363 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:28.991061 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1efce1ebdf1425396c2babae74fe6ce', 'request_id': 'req-47bd9fc5-b450-449e-bc55-500e8df6240e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:28.998972 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1efce1ebdf1425396c2babae74fe6ce', 'request_id': 'req-47bd9fc5-b450-449e-bc55-500e8df6240e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:29.052598 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1efce1ebdf1425396c2babae74fe6ce', 'request_id': 'req-47bd9fc5-b450-449e-bc55-500e8df6240e'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:29.064272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] The token cache is being invalidate because project a1efce1ebdf1425396c2babae74fe6ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:52:29.064547 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:29.065165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-47bd9fc5-b450-449e-bc55-500e8df6240e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:29.067013 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1468/2898] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:52:28 2022] DELETE /identity/v3/projects/a1efce1ebdf1425396c2babae74fe6ce => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:29.072428 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5932749-e2ef-41bb-a2d7-b6295d73108e None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:29.072957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5932749-e2ef-41bb-a2d7-b6295d73108e None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:29.072957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5932749-e2ef-41bb-a2d7-b6295d73108e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:29.096189 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-f5932749-e2ef-41bb-a2d7-b6295d73108e None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:29.097464 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-f5932749-e2ef-41bb-a2d7-b6295d73108e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:29.161308 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1431/2899] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:29.169698 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d9391-0df0-4b96-80b5-6eaee237b91c None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:29.169989 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d9391-0df0-4b96-80b5-6eaee237b91c None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:29.170301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d9391-0df0-4b96-80b5-6eaee237b91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:29.190463 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-569d9391-0df0-4b96-80b5-6eaee237b91c None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:29.191685 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-569d9391-0df0-4b96-80b5-6eaee237b91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:29.200244 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1469/2900] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:29.204325 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5932749-e2ef-41bb-a2d7-b6295d73108e None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:29.216860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5932749-e2ef-41bb-a2d7-b6295d73108e None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:29.224279 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6df15-f242-4774-9858-c8868b7660de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:29.224814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6df15-f242-4774-9858-c8868b7660de admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:29.225182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6df15-f242-4774-9858-c8868b7660de admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:29.225529 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6df15-f242-4774-9858-c8868b7660de admin admin] PATH_INFO: `/v3/domains` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:29.226544 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6df15-f242-4774-9858-c8868b7660de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81480) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 05:52:29.238921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6df15-f242-4774-9858-c8868b7660de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:29.238921 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6df15-f242-4774-9858-c8868b7660de admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:29.243313 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1432/2901] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 05:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 05:52:29.247529 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d9391-0df0-4b96-80b5-6eaee237b91c None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:29.257179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d9391-0df0-4b96-80b5-6eaee237b91c None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:29.266583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ef955-3c1b-479e-b23c-6d1ffee338e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:29.266583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ef955-3c1b-479e-b23c-6d1ffee338e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:29.266583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ef955-3c1b-479e-b23c-6d1ffee338e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:29.266583 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ef955-3c1b-479e-b23c-6d1ffee338e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:29.267049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ef955-3c1b-479e-b23c-6d1ffee338e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:29.267049 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ef955-3c1b-479e-b23c-6d1ffee338e2 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:29.377729 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1470/2902] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 05:52:29 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:52:29.379085 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6df15-f242-4774-9858-c8868b7660de admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:29.390696 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6df15-f242-4774-9858-c8868b7660de admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:29.468577 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac2486-2999-44c9-9b91-a47cad21a663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:29.469188 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac2486-2999-44c9-9b91-a47cad21a663 admin admin] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:29.469462 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac2486-2999-44c9-9b91-a47cad21a663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:29.469736 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac2486-2999-44c9-9b91-a47cad21a663 admin admin] PATH_INFO: `/v3/users` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:29.470391 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac2486-2999-44c9-9b91-a47cad21a663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:29.472281 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac2486-2999-44c9-9b91-a47cad21a663 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:29.570585 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:29.574604 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1433/2903] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 05:52:29 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 05:52:29.578791 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ef955-3c1b-479e-b23c-6d1ffee338e2 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:29.589631 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ef955-3c1b-479e-b23c-6d1ffee338e2 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:29.594106 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdffe3-d1fe-40e3-89b3-fce3dc8c91e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:29.596275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdffe3-d1fe-40e3-89b3-fce3dc8c91e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:29.596275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdffe3-d1fe-40e3-89b3-fce3dc8c91e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:29.596275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdffe3-d1fe-40e3-89b3-fce3dc8c91e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:29.596275 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdffe3-d1fe-40e3-89b3-fce3dc8c91e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:29.597453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdffe3-d1fe-40e3-89b3-fce3dc8c91e6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:29.608828 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1471/2904] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 05:52:29.613100 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac2486-2999-44c9-9b91-a47cad21a663 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:29.623583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac2486-2999-44c9-9b91-a47cad21a663 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:29.629108 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c22da-77cf-4d0b-956a-8f0fa5563af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:29.629742 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c22da-77cf-4d0b-956a-8f0fa5563af2 admin admin] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:29.630021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c22da-77cf-4d0b-956a-8f0fa5563af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:29.630329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c22da-77cf-4d0b-956a-8f0fa5563af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:29.630969 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c22da-77cf-4d0b-956a-8f0fa5563af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:29.632542 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c22da-77cf-4d0b-956a-8f0fa5563af2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:29.642697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1434/2905] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 05:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 05:52:29.647154 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdffe3-d1fe-40e3-89b3-fce3dc8c91e6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:29.657859 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdffe3-d1fe-40e3-89b3-fce3dc8c91e6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:29.662459 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a312563-50a7-41af-8b13-a57c33ff4556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:29.663124 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a312563-50a7-41af-8b13-a57c33ff4556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:29.663404 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a312563-50a7-41af-8b13-a57c33ff4556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:29.663680 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a312563-50a7-41af-8b13-a57c33ff4556 admin admin] PATH_INFO: `/v3/projects/169f5575f7bf4d25874563d22cb6fc14/users/ecbae3e0fc3f4410b4483449eba44277/roles/1e04fdc0cc334fd7a733a1490a194e37` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:29.680699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a312563-50a7-41af-8b13-a57c33ff4556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169f5575f7bf4d25874563d22cb6fc14, user_id=ecbae3e0fc3f4410b4483449eba44277, role_id=1e04fdc0cc334fd7a733a1490a194e37)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:29.683408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a312563-50a7-41af-8b13-a57c33ff4556 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:29.811627 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1472/2906] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 05:52:29 2022] PUT /identity/v3/projects/169f5575f7bf4d25874563d22cb6fc14/users/ecbae3e0fc3f4410b4483449eba44277/roles/1e04fdc0cc334fd7a733a1490a194e37 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:29.821460 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff325c-daee-4065-8bc8-36c314febb48 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:29.821747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff325c-daee-4065-8bc8-36c314febb48 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:29.821957 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff325c-daee-4065-8bc8-36c314febb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:29.855062 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-2fff325c-daee-4065-8bc8-36c314febb48 None None] MFA Rules not processed for user `ecbae3e0fc3f4410b4483449eba44277`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:29.856332 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2fff325c-daee-4065-8bc8-36c314febb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:29.932021 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1435/2907] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:29 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:29.943912 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a312563-50a7-41af-8b13-a57c33ff4556 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:29.946166 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-0a312563-50a7-41af-8b13-a57c33ff4556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:29.962738 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:30.022728 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a312563-50a7-41af-8b13-a57c33ff4556 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:30.027177 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e70d9-1c96-44ab-af5c-f898e8739e03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:30.028026 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e70d9-1c96-44ab-af5c-f898e8739e03 service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:30.028186 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e70d9-1c96-44ab-af5c-f898e8739e03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:30.028330 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e70d9-1c96-44ab-af5c-f898e8739e03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:30.036835 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e70d9-1c96-44ab-af5c-f898e8739e03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:30.038307 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e70d9-1c96-44ab-af5c-f898e8739e03 service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:30.047481 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1473/2908] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:30.769182 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79816e6-c8f2-42f7-8f14-4725d5843f5f None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:30.769737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79816e6-c8f2-42f7-8f14-4725d5843f5f None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:30.769737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79816e6-c8f2-42f7-8f14-4725d5843f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:30.788305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-d79816e6-c8f2-42f7-8f14-4725d5843f5f None None] MFA Rules not processed for user `92f4eb565bdf49f197d6b2b4a3f17cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:30.790046 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-d79816e6-c8f2-42f7-8f14-4725d5843f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:30.845885 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1436/2909] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:38.957039 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e70d9-1c96-44ab-af5c-f898e8739e03 service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:39.024936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e70d9-1c96-44ab-af5c-f898e8739e03 service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:39.030327 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:39.030956 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:39.031255 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:39.031526 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] PATH_INFO: `/v3/users/ecbae3e0fc3f4410b4483449eba44277` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:39.032640 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecbae3e0fc3f4410b4483449eba44277)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:39.034588 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:39.204812 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecbae3e0fc3f4410b4483449eba44277'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:39.218301 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecbae3e0fc3f4410b4483449eba44277'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:39.219031 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecbae3e0fc3f4410b4483449eba44277'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:39.315362 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecbae3e0fc3f4410b4483449eba44277'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:39.316007 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecbae3e0fc3f4410b4483449eba44277'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:39.329438 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1474/2910] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:52:38 2022] DELETE /identity/v3/users/ecbae3e0fc3f4410b4483449eba44277 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:39.334678 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f00fc-92bf-4b87-8ceb-184cb38d1e30 None None] REQUEST_METHOD: `POST` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:39.335219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f00fc-92bf-4b87-8ceb-184cb38d1e30 None None] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:39.335219 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f00fc-92bf-4b87-8ceb-184cb38d1e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:39.355459 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.auth.core [None req-870f00fc-92bf-4b87-8ceb-184cb38d1e30 None None] MFA Rules not processed for user `4444aa1a4685411fad68fe8a8f2d8f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81480) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:52:39.356681 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-870f00fc-92bf-4b87-8ceb-184cb38d1e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:39.414275 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1437/2911] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:52:39.424472 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:39.425863 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:39.462162 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:39.484671 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fccf9-e18a-491b-996e-6abb48cc9598 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:39.488376 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585f31e-b706-47c1-99a9-f25422666f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:39.488977 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585f31e-b706-47c1-99a9-f25422666f94 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:39.489256 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585f31e-b706-47c1-99a9-f25422666f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:39.489528 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585f31e-b706-47c1-99a9-f25422666f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:39.498967 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585f31e-b706-47c1-99a9-f25422666f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:39.506115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585f31e-b706-47c1-99a9-f25422666f94 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:39.509741 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1475/2912] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:39.877903 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f00fc-92bf-4b87-8ceb-184cb38d1e30 None None] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:39.879413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-870f00fc-92bf-4b87-8ceb-184cb38d1e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:39.897583 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f00fc-92bf-4b87-8ceb-184cb38d1e30 None None] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:39.902578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:39.903238 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:39.903516 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:39.903786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] PATH_INFO: `/v3/projects/169f5575f7bf4d25874563d22cb6fc14` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:39.905074 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=169f5575f7bf4d25874563d22cb6fc14)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:39.906995 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:39.925013 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=100 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:40.033593 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169f5575f7bf4d25874563d22cb6fc14', 'request_id': 'req-a4936308-7659-4742-af7f-c3e7b0b4bca5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:40.042398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169f5575f7bf4d25874563d22cb6fc14', 'request_id': 'req-a4936308-7659-4742-af7f-c3e7b0b4bca5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:40.098155 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169f5575f7bf4d25874563d22cb6fc14', 'request_id': 'req-a4936308-7659-4742-af7f-c3e7b0b4bca5'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:40.111502 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] The token cache is being invalidate because project 169f5575f7bf4d25874563d22cb6fc14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:52:40.111877 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:40.112517 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:52:40.114072 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1438/2913] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:52:39 2022] DELETE /identity/v3/projects/169f5575f7bf4d25874563d22cb6fc14 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:52:48.505999 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585f31e-b706-47c1-99a9-f25422666f94 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:48.507453 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c585f31e-b706-47c1-99a9-f25422666f94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:48.566941 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585f31e-b706-47c1-99a9-f25422666f94 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:48.616710 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14bb1d-8494-4a5f-8fb8-74b04338fe5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:48.617320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14bb1d-8494-4a5f-8fb8-74b04338fe5e service cinder] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:48.617596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14bb1d-8494-4a5f-8fb8-74b04338fe5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:48.617864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14bb1d-8494-4a5f-8fb8-74b04338fe5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:48.619620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ae14bb1d-8494-4a5f-8fb8-74b04338fe5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:48.688953 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14bb1d-8494-4a5f-8fb8-74b04338fe5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:48.690430 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14bb1d-8494-4a5f-8fb8-74b04338fe5e service cinder] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:48.701596 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1476/2914] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:49.645338 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:52:49.646932 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:49.705927 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4936308-7659-4742-af7f-c3e7b0b4bca5 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:52:49.709064 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c12e6d-8ba2-4894-8ac0-402fcf313ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:52:49.709796 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c12e6d-8ba2-4894-8ac0-402fcf313ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:52:49.710213 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c12e6d-8ba2-4894-8ac0-402fcf313ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:52:49.710608 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c12e6d-8ba2-4894-8ac0-402fcf313ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:52:49.712445 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-90c12e6d-8ba2-4894-8ac0-402fcf313ee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:52:49.726500 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c12e6d-8ba2-4894-8ac0-402fcf313ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:52:49.727909 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c12e6d-8ba2-4894-8ac0-402fcf313ee0 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:52:49.740226 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1439/2915] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:52:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:52:58.696699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=85 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:52:59.735973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:53:06.240560 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14bb1d-8494-4a5f-8fb8-74b04338fe5e service cinder] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:53:06.241966 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ae14bb1d-8494-4a5f-8fb8-74b04338fe5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:06.253348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14bb1d-8494-4a5f-8fb8-74b04338fe5e service cinder] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:53:06.256961 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ce9e3-98bf-4cbb-bf33-8b3352d93ffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:53:06.257827 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ce9e3-98bf-4cbb-bf33-8b3352d93ffb service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:53:06.258108 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ce9e3-98bf-4cbb-bf33-8b3352d93ffb service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:53:06.258408 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ce9e3-98bf-4cbb-bf33-8b3352d93ffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:53:06.260083 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-592ce9e3-98bf-4cbb-bf33-8b3352d93ffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:06.321701 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ce9e3-98bf-4cbb-bf33-8b3352d93ffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:53:06.323176 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ce9e3-98bf-4cbb-bf33-8b3352d93ffb service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:53:06.332605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1477/2916] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:53:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:53:11.732971 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c12e6d-8ba2-4894-8ac0-402fcf313ee0 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:53:11.739455 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-90c12e6d-8ba2-4894-8ac0-402fcf313ee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:11.746249 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c12e6d-8ba2-4894-8ac0-402fcf313ee0 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:53:11.749973 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3777a69-71f5-4855-8d21-3236df6cc9d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '131afaf047d34e88842fffd0d0b76242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:53:11.750625 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3777a69-71f5-4855-8d21-3236df6cc9d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:53:11.750901 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3777a69-71f5-4855-8d21-3236df6cc9d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:53:11.751174 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3777a69-71f5-4855-8d21-3236df6cc9d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:53:11.752873 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-c3777a69-71f5-4855-8d21-3236df6cc9d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:11.812840 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3777a69-71f5-4855-8d21-3236df6cc9d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:53:11.814317 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3777a69-71f5-4855-8d21-3236df6cc9d1 service cinder] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:53:11.824413 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1440/2917] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:53:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:53:16.328465 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=30 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:53:20.265864 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ded71-d31f-4287-8e98-58ff4dbc52e8 None None] REQUEST_METHOD: `POST` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:53:20.266486 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ded71-d31f-4287-8e98-58ff4dbc52e8 None None] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:53:20.266486 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ded71-d31f-4287-8e98-58ff4dbc52e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:53:20.285709 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.auth.core [None req-2e4ded71-d31f-4287-8e98-58ff4dbc52e8 None None] MFA Rules not processed for user `0c0c0a25340a451b970cc3de8bc05806`. Rule list: `[]` (Enabled: `True`). {{(pid=81481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 05:53:20.286936 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2e4ded71-d31f-4287-8e98-58ff4dbc52e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:20.294389 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1478/2918] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 05:53:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 05:53:20.604786 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3777a69-71f5-4855-8d21-3236df6cc9d1 service cinder] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:53:20.618548 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3777a69-71f5-4855-8d21-3236df6cc9d1 service cinder] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:53:20.622124 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e541cbd-5456-46d6-9ba8-0fad17be01d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:53:20.622745 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e541cbd-5456-46d6-9ba8-0fad17be01d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:53:20.623027 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e541cbd-5456-46d6-9ba8-0fad17be01d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:53:20.623331 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e541cbd-5456-46d6-9ba8-0fad17be01d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:53:20.625035 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9e541cbd-5456-46d6-9ba8-0fad17be01d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:20.634374 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e541cbd-5456-46d6-9ba8-0fad17be01d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:53:20.636285 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e541cbd-5456-46d6-9ba8-0fad17be01d4 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:53:20.644130 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1441/2919] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:53:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:53:21.311436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ded71-d31f-4287-8e98-58ff4dbc52e8 None None] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:53:21.312911 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-2e4ded71-d31f-4287-8e98-58ff4dbc52e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:21.384298 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ded71-d31f-4287-8e98-58ff4dbc52e8 None None] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:53:21.389785 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388bd9b-03d5-452b-928d-63a58fc23816 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:53:21.390423 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388bd9b-03d5-452b-928d-63a58fc23816 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:53:21.390706 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388bd9b-03d5-452b-928d-63a58fc23816 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:53:21.390982 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388bd9b-03d5-452b-928d-63a58fc23816 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:53:21.392700 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1388bd9b-03d5-452b-928d-63a58fc23816 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:21.404604 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388bd9b-03d5-452b-928d-63a58fc23816 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:53:21.406213 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388bd9b-03d5-452b-928d-63a58fc23816 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:53:21.416179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1479/2920] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:53:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:53:24.702559 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e541cbd-5456-46d6-9ba8-0fad17be01d4 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:53:24.711895 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e541cbd-5456-46d6-9ba8-0fad17be01d4 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:53:24.716247 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ac94e-fcd5-4284-9b7e-94e47e534500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:53:24.719439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ac94e-fcd5-4284-9b7e-94e47e534500 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:53:24.719439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ac94e-fcd5-4284-9b7e-94e47e534500 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:53:24.719439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ac94e-fcd5-4284-9b7e-94e47e534500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:53:24.719439 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-265ac94e-fcd5-4284-9b7e-94e47e534500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:24.729145 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ac94e-fcd5-4284-9b7e-94e47e534500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:53:24.730624 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ac94e-fcd5-4284-9b7e-94e47e534500 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:53:24.738649 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1442/2921] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:53:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:53:25.585948 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388bd9b-03d5-452b-928d-63a58fc23816 service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:53:25.587576 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-1388bd9b-03d5-452b-928d-63a58fc23816 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:25.603487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388bd9b-03d5-452b-928d-63a58fc23816 service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:53:25.609889 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f202fad-5c73-42c0-934a-0bbc693cf66a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:53:25.610525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f202fad-5c73-42c0-934a-0bbc693cf66a service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:53:25.610807 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f202fad-5c73-42c0-934a-0bbc693cf66a service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:53:25.611081 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f202fad-5c73-42c0-934a-0bbc693cf66a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:53:25.613116 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-7f202fad-5c73-42c0-934a-0bbc693cf66a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:25.626436 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f202fad-5c73-42c0-934a-0bbc693cf66a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:53:25.628248 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f202fad-5c73-42c0-934a-0bbc693cf66a service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:53:25.641488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1480/2922] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:53:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:53:34.734855 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=40 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:53:35.634401 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=41 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:53:40.620446 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ac94e-fcd5-4284-9b7e-94e47e534500 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:53:40.628993 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ac94e-fcd5-4284-9b7e-94e47e534500 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:53:40.632578 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ff209-13b7-4f2c-a4d3-37a4d6f95945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:53:40.633175 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ff209-13b7-4f2c-a4d3-37a4d6f95945 service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:53:40.633454 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ff209-13b7-4f2c-a4d3-37a4d6f95945 service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:53:40.633724 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ff209-13b7-4f2c-a4d3-37a4d6f95945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:53:40.635451 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9f3ff209-13b7-4f2c-a4d3-37a4d6f95945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:40.697868 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ff209-13b7-4f2c-a4d3-37a4d6f95945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:53:40.699318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ff209-13b7-4f2c-a4d3-37a4d6f95945 service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:53:40.708611 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1443/2923] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:53:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:53:42.517272 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f202fad-5c73-42c0-934a-0bbc693cf66a service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:53:42.529254 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f202fad-5c73-42c0-934a-0bbc693cf66a service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:53:42.535148 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209bc81-0b49-498a-b6b8-4d8780cd2c83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:53:42.535511 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209bc81-0b49-498a-b6b8-4d8780cd2c83 service nova] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:53:42.535793 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209bc81-0b49-498a-b6b8-4d8780cd2c83 service nova] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:53:42.536066 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209bc81-0b49-498a-b6b8-4d8780cd2c83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:53:42.537849 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-d209bc81-0b49-498a-b6b8-4d8780cd2c83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:53:42.550133 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209bc81-0b49-498a-b6b8-4d8780cd2c83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:53:42.551739 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209bc81-0b49-498a-b6b8-4d8780cd2c83 service nova] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:53:42.562446 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1481/2924] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:53:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:53:50.704340 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=30 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:53:52.558595 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=6 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:55:20.739478 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ff209-13b7-4f2c-a4d3-37a4d6f95945 service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:55:20.750788 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ff209-13b7-4f2c-a4d3-37a4d6f95945 service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:55:20.754975 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080edbb-fcdc-479b-b9ae-cf337de3dc3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:55:20.756033 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080edbb-fcdc-479b-b9ae-cf337de3dc3f service neutron] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:55:20.756312 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080edbb-fcdc-479b-b9ae-cf337de3dc3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:55:20.756582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080edbb-fcdc-479b-b9ae-cf337de3dc3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:55:20.758737 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-9080edbb-fcdc-479b-b9ae-cf337de3dc3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:55:20.772025 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080edbb-fcdc-479b-b9ae-cf337de3dc3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:55:20.773787 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080edbb-fcdc-479b-b9ae-cf337de3dc3f service neutron] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:55:20.783851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1444/2925] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:55:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:55:20.865744 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209bc81-0b49-498a-b6b8-4d8780cd2c83 service nova] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:55:20.878188 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209bc81-0b49-498a-b6b8-4d8780cd2c83 service nova] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:55:20.881843 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf51e65-f36a-45e2-be16-7a0defb6a8a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:55:20.882494 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf51e65-f36a-45e2-be16-7a0defb6a8a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:55:20.882770 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf51e65-f36a-45e2-be16-7a0defb6a8a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:55:20.883042 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf51e65-f36a-45e2-be16-7a0defb6a8a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:55:20.885021 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-bcf51e65-f36a-45e2-be16-7a0defb6a8a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:55:20.901690 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf51e65-f36a-45e2-be16-7a0defb6a8a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:55:20.903153 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf51e65-f36a-45e2-be16-7a0defb6a8a1 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:55:20.914946 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1482/2926] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:55:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:55:23.779844 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080edbb-fcdc-479b-b9ae-cf337de3dc3f service neutron] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:55:23.793218 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080edbb-fcdc-479b-b9ae-cf337de3dc3f service neutron] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:55:23.799193 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4bc81-d569-40f9-a103-733878f1e6c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fba4f0473f4c4d3bb2d50b09f3de9837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:55:23.801785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4bc81-d569-40f9-a103-733878f1e6c5 service nova] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:55:23.801785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4bc81-d569-40f9-a103-733878f1e6c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:55:23.801785 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4bc81-d569-40f9-a103-733878f1e6c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:55:23.802380 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-59f4bc81-d569-40f9-a103-733878f1e6c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:55:23.815282 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4bc81-d569-40f9-a103-733878f1e6c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:55:23.816987 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4bc81-d569-40f9-a103-733878f1e6c5 service nova] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:55:23.829084 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1445/2927] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:55:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:55:27.059320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf51e65-f36a-45e2-be16-7a0defb6a8a1 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:55:27.068622 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf51e65-f36a-45e2-be16-7a0defb6a8a1 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:55:27.072584 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c66afe-7c2e-47f4-bed2-843803c91b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a0feaf83ee14071806f36d01c36b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:55:27.073207 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c66afe-7c2e-47f4-bed2-843803c91b11 service neutron] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:55:27.073483 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c66afe-7c2e-47f4-bed2-843803c91b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:55:27.073749 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c66afe-7c2e-47f4-bed2-843803c91b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:55:27.075493 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-c4c66afe-7c2e-47f4-bed2-843803c91b11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:55:27.137600 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c66afe-7c2e-47f4-bed2-843803c91b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:55:27.139058 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c66afe-7c2e-47f4-bed2-843803c91b11 service neutron] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:55:27.148510 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1483/2928] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:55:30.486684 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4bc81-d569-40f9-a103-733878f1e6c5 service nova] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:55:30.498890 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4bc81-d569-40f9-a103-733878f1e6c5 service nova] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:55:30.506783 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02748bc-a97d-4a0f-9fa1-88e1a013b6f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:55:30.507705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02748bc-a97d-4a0f-9fa1-88e1a013b6f3 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:55:30.509305 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02748bc-a97d-4a0f-9fa1-88e1a013b6f3 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:55:30.509926 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02748bc-a97d-4a0f-9fa1-88e1a013b6f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:55:30.512495 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a02748bc-a97d-4a0f-9fa1-88e1a013b6f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:55:30.525713 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02748bc-a97d-4a0f-9fa1-88e1a013b6f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:55:30.527594 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02748bc-a97d-4a0f-9fa1-88e1a013b6f3 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:55:30.538809 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1446/2929] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:55:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:55:34.064868 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c66afe-7c2e-47f4-bed2-843803c91b11 service neutron] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:55:34.079539 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c66afe-7c2e-47f4-bed2-843803c91b11 service neutron] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:55:34.085043 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b199354-d709-4c1f-afca-e2023646f44c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:55:34.086194 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b199354-d709-4c1f-afca-e2023646f44c service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:55:34.086713 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b199354-d709-4c1f-afca-e2023646f44c service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:55:34.087202 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b199354-d709-4c1f-afca-e2023646f44c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:55:34.089230 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-3b199354-d709-4c1f-afca-e2023646f44c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:55:34.102305 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b199354-d709-4c1f-afca-e2023646f44c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:55:34.104015 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b199354-d709-4c1f-afca-e2023646f44c service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:55:34.115488 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1484/2930] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:55:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 05:55:34.192706 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02748bc-a97d-4a0f-9fa1-88e1a013b6f3 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:55:34.194132 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a02748bc-a97d-4a0f-9fa1-88e1a013b6f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:55:34.210560 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02748bc-a97d-4a0f-9fa1-88e1a013b6f3 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:55:34.215028 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:55:34.215666 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:55:34.215961 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:55:34.216222 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] PATH_INFO: `/v3/users/0c0c0a25340a451b970cc3de8bc05806` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:55:34.217337 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0c0a25340a451b970cc3de8bc05806)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:55:34.220065 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:55:34.424170 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0c0a25340a451b970cc3de8bc05806'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:34.424827 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0c0a25340a451b970cc3de8bc05806'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:34.437131 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0c0a25340a451b970cc3de8bc05806'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:34.535747 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0c0a25340a451b970cc3de8bc05806'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:34.536361 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0c0a25340a451b970cc3de8bc05806'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:34.549073 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1447/2931] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:55:34 2022] DELETE /identity/v3/users/0c0c0a25340a451b970cc3de8bc05806 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:55:34.772806 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b199354-d709-4c1f-afca-e2023646f44c service placement] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:55:34.784077 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b199354-d709-4c1f-afca-e2023646f44c service placement] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:55:34.847320 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:55:34.848339 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:55:34.848832 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:55:34.849173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] PATH_INFO: `/v3/projects/45a5869786d843fe9fb5cdb6b3596c2f` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:55:34.850456 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a5869786d843fe9fb5cdb6b3596c2f)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:55:34.852400 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:55:34.938165 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a5869786d843fe9fb5cdb6b3596c2f', 'request_id': 'req-30849bb9-ba14-4369-ade6-3936d018b78d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:34.944717 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a5869786d843fe9fb5cdb6b3596c2f', 'request_id': 'req-30849bb9-ba14-4369-ade6-3936d018b78d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:34.987719 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a5869786d843fe9fb5cdb6b3596c2f', 'request_id': 'req-30849bb9-ba14-4369-ade6-3936d018b78d'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:34.994286 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:55:34.998780 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] The token cache is being invalidate because project 45a5869786d843fe9fb5cdb6b3596c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:55:35.004722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.004722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.004722 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1485/2932] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:55:34 2022] DELETE /identity/v3/projects/45a5869786d843fe9fb5cdb6b3596c2f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:55:35.006049 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:55:35.007584 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:55:35.020531 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2761df6-c03a-41b6-acc1-dac7882640ab admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:55:35.076697 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:55:35.077256 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:55:35.077541 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:55:35.077814 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] PATH_INFO: `/v3/users/3e47eb113db74cc0a40906cca4d2f403` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:55:35.079250 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e47eb113db74cc0a40906cca4d2f403)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:55:35.081140 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:55:35.250614 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:55:35.260055 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e47eb113db74cc0a40906cca4d2f403'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.260770 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e47eb113db74cc0a40906cca4d2f403'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.272481 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e47eb113db74cc0a40906cca4d2f403'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.324568 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e47eb113db74cc0a40906cca4d2f403'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.325181 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e47eb113db74cc0a40906cca4d2f403'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.339534 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1448/2933] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:55:35 2022] DELETE /identity/v3/users/3e47eb113db74cc0a40906cca4d2f403 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:55:35.605115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:55:35.606620 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:55:35.622779 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30849bb9-ba14-4369-ade6-3936d018b78d admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:55:35.680087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:55:35.680742 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:55:35.681016 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:55:35.681286 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] PATH_INFO: `/v3/projects/b1c0de68f6474ce2b0ec27fd91a5a166` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:55:35.682444 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c0de68f6474ce2b0ec27fd91a5a166)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:55:35.684348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:55:35.787487 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c0de68f6474ce2b0ec27fd91a5a166', 'request_id': 'req-31ae0069-113a-4aaa-acdf-e47f522318d6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.793943 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c0de68f6474ce2b0ec27fd91a5a166', 'request_id': 'req-31ae0069-113a-4aaa-acdf-e47f522318d6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.850358 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c0de68f6474ce2b0ec27fd91a5a166', 'request_id': 'req-31ae0069-113a-4aaa-acdf-e47f522318d6'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.861887 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] The token cache is being invalidate because project b1c0de68f6474ce2b0ec27fd91a5a166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:55:35.862346 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.863115 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:55:35.864605 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1486/2934] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:55:35 2022] DELETE /identity/v3/projects/b1c0de68f6474ce2b0ec27fd91a5a166 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:55:45.376398 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:55:45.858616 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:56:58.303898 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:56:58.305306 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:56:58.373122 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d490bc-2ec4-4268-8993-6d9c9bc3e744 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:56:58.433683 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c999685-3187-47c3-9322-6f101360d8f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:56:58.434329 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c999685-3187-47c3-9322-6f101360d8f4 service placement] REQUEST_METHOD: `GET` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:56:58.434705 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c999685-3187-47c3-9322-6f101360d8f4 service placement] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:56:58.434978 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c999685-3187-47c3-9322-6f101360d8f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:56:58.436794 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-2c999685-3187-47c3-9322-6f101360d8f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:56:58.448239 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c999685-3187-47c3-9322-6f101360d8f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:56:58.449851 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c999685-3187-47c3-9322-6f101360d8f4 service placement] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:56:58.459860 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1449/2935] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 05:56:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 05:57:07.923978 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:57:07.925439 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:57:07.997173 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae0069-113a-4aaa-acdf-e47f522318d6 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:57:08.001638 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:57:08.002323 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:57:08.003101 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:57:08.003375 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] PATH_INFO: `/v3/users/92f4eb565bdf49f197d6b2b4a3f17cc8` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:57:08.004525 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] RBAC: Authorizing `identity:delete_user(user_id=92f4eb565bdf49f197d6b2b4a3f17cc8)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:57:08.006482 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:57:08.164522 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92f4eb565bdf49f197d6b2b4a3f17cc8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:08.176955 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92f4eb565bdf49f197d6b2b4a3f17cc8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:08.177630 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92f4eb565bdf49f197d6b2b4a3f17cc8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:08.248895 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92f4eb565bdf49f197d6b2b4a3f17cc8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:08.249448 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92f4eb565bdf49f197d6b2b4a3f17cc8'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:08.262676 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1487/2936] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:57:07 2022] DELETE /identity/v3/users/92f4eb565bdf49f197d6b2b4a3f17cc8 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:57:08.455771 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=58 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:57:08.508281 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c999685-3187-47c3-9322-6f101360d8f4 service placement] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:57:08.518582 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c999685-3187-47c3-9322-6f101360d8f4 service placement] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:57:08.574811 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:57:08.575433 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:57:08.575725 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:57:08.575990 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] PATH_INFO: `/v3/projects/c225dfb273d64b3e94f7dae792e4cd9d` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:57:08.577103 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c225dfb273d64b3e94f7dae792e4cd9d)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:57:08.579013 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:57:08.671992 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c225dfb273d64b3e94f7dae792e4cd9d', 'request_id': 'req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:08.678829 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c225dfb273d64b3e94f7dae792e4cd9d', 'request_id': 'req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:08.740513 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c225dfb273d64b3e94f7dae792e4cd9d', 'request_id': 'req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:08.752229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] The token cache is being invalidate because project c225dfb273d64b3e94f7dae792e4cd9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:57:08.752229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:08.752229 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:08.753473 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1450/2937] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:57:08 2022] DELETE /identity/v3/projects/c225dfb273d64b3e94f7dae792e4cd9d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:57:08.757470 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:57:08.758896 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:57:08.771705 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448601a-7060-4af4-ae87-a737bd98927e admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:57:08.828103 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:57:08.829087 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:57:08.829365 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:57:08.829634 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] PATH_INFO: `/v3/users/237d6b5f9a2e4ed1966caf70e88bcf68` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:57:08.830862 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=237d6b5f9a2e4ed1966caf70e88bcf68)` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:57:08.832856 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:57:08.854592 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:57:09.080347 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '237d6b5f9a2e4ed1966caf70e88bcf68'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:09.092699 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '237d6b5f9a2e4ed1966caf70e88bcf68'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:09.093366 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '237d6b5f9a2e4ed1966caf70e88bcf68'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:09.168584 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '237d6b5f9a2e4ed1966caf70e88bcf68'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:09.169192 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.notifications [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '237d6b5f9a2e4ed1966caf70e88bcf68'} {{(pid=81481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:09.181355 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1488/2938] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 05:57:08 2022] DELETE /identity/v3/users/237d6b5f9a2e4ed1966caf70e88bcf68 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:57:09.427677 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] Authenticating user token {{(pid=81480) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 05:57:09.429177 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.fernet_utils [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81480) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 05:57:09.445040 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c844dbf-b7ff-4ea6-998d-c4824ef286e2 admin admin] Validating token access rules against request {{(pid=81480) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 05:57:09.501519 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4444aa1a4685411fad68fe8a8f2d8f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c2b6383659b48fcabc56888b6bfb190', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81480) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 05:57:09.502216 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 05:57:09.503008 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 05:57:09.503291 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] PATH_INFO: `/v3/projects/a460d3b4dfa14b3ba28c577fa7c30f60` {{(pid=81480) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 05:57:09.504412 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a460d3b4dfa14b3ba28c577fa7c30f60)` {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 05:57:09.506388 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] RBAC: Authorization granted {{(pid=81480) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 05:57:09.586318 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:57:09.597947 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a460d3b4dfa14b3ba28c577fa7c30f60', 'request_id': 'req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:09.604209 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a460d3b4dfa14b3ba28c577fa7c30f60', 'request_id': 'req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:09.671911 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a460d3b4dfa14b3ba28c577fa7c30f60', 'request_id': 'req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8'} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:09.682617 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] The token cache is being invalidate because project a460d3b4dfa14b3ba28c577fa7c30f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81480) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 05:57:09.682908 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:09.683532 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG keystone.notifications [None req-00649bb4-2a2d-4d23-8839-ac1e6d95e5f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81480) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 05:57:09.685011 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: [pid: 81480|app: 0|req: 1451/2939] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 05:57:09 2022] DELETE /identity/v3/projects/a460d3b4dfa14b3ba28c577fa7c30f60 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 05:57:19.177959 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 05:57:19.751748 dv-cinder-849143-10-iscsi devstack@keystone.service[81480]: DEBUG dbcounter [-] [81480] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81480) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 06:02:06.306300 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] Authenticating user token {{(pid=81481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 06:02:06.307697 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 06:02:06.443899 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec451112-0306-4e02-b3d1-5b5ce4e31903 admin admin] Validating token access rules against request {{(pid=81481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 06:02:06.502161 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c465743-251c-496b-988e-1534e06d0bf5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e95901cce5d448acb2571976163868a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e74d59a086274094806c1c6cf2964451', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 06:02:06.502787 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c465743-251c-496b-988e-1534e06d0bf5 service placement] REQUEST_METHOD: `GET` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 06:02:06.503070 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c465743-251c-496b-988e-1534e06d0bf5 service placement] SCRIPT_NAME: `/identity` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 06:02:06.503348 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c465743-251c-496b-988e-1534e06d0bf5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 06:02:06.505099 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.fernet_utils [None req-4c465743-251c-496b-988e-1534e06d0bf5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 06:02:06.516088 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c465743-251c-496b-988e-1534e06d0bf5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 06:02:06.518179 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c465743-251c-496b-988e-1534e06d0bf5 service placement] RBAC: Authorization granted {{(pid=81481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 06:02:06.528076 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: [pid: 81481|app: 0|req: 1489/2940] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 06:02:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 06:02:16.523751 dv-cinder-849143-10-iscsi devstack@keystone.service[81481]: DEBUG dbcounter [-] [81481] Writing DB stats keystone:SELECT=89 {{(pid=81481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}